DEV Community: AI Explore

samkhya v1.0: Plug Claude, GPT-4o-mini, or Local Ollama Into Your SQL Query Optimizer

AI Explore — Sun, 28 Jun 2026 18:45:39 +0000

TL;DR

samkhya v1.0 ships the first LLM-pluggable corrector backend for an embedded SQL query optimizer. Plug Anthropic Claude (claude-opus-4-7, claude-sonnet-4-6), OpenAI GPT-4o-mini, or local Ollama (llama3.2:1b) into the cardinality-estimation slot of DataFusion, DuckDB, or Polars via a 4-line HTTP wire contract. Two reference servers ship in the box: Python FastAPI (canonical) and Node TypeScript (broader operator appeal). It's all wrapped in a provable safety envelope so a hallucinating LLM cannot make your plan worse than the engine's native estimate. Apache-2.0, sole author. 13-crate Rust workspace.

cargo add samkhya-core --features llm_http · pip install samkhya
github.com/singhpratech/samkhya— 10 crates on crates.io, Python wheel on PyPI
Companion technical paper + gpudb sibling release

Measured numbers, all reported honestly. 40.95× LpJoinBound tightness over AGM on the star-5 join family at p=1 (Wilcoxon p=1.73×10⁻⁶, all 30 cells dominated). LLM transport-floor P95 0.07–0.11 ms across batch sizes 1/4/8/16/32— H1-A PASS, the LLM plug works. JOB-Slow head-to-head vs unmodified DataFusion 46 came in at 1.038× geomean wallclock (BCa 95% CI [1.026, 1.056], Wilcoxon p=3×10⁻⁶, 17 wins / 38 ties / 0 losses)— pre-registered ≥1.35× target falsified on magnitude. Never-regress holds.

Honestly projected, not yet measured. Live-LLM end-to-end latency cells (Claude, GPT-4o-mini, Ollama) are documented as PROJECTED pending API-key budget and a 30-trial campaign run. The mechanism is shipped and measured at the transport floor; the headline magnitudes are paper-projections sized from public per-token latencies.

Every embedded analytical engine on your laptop right now is committing the same crime in slow motion. DuckDB does it. DataFusion does it. Polars does it. ClickHouse-local does it. My own GPU engine gpudb does it. The crime is this: every time the process dies, eight years of incredible cardinality-estimation research dies with it, and the next session starts over from zero. The HyperLogLog your midnight ELT job built? Gone. The Bloom filter that knew exactly which customer IDs lived in that 400 GB Parquet partition? Gone. The histogram that took twelve seconds to compute over the join key? Gone. The optimizer wakes up at 9 a.m. with no memory of anything it ever learned and goes scanning the same columns again.

That waste isn't a bug. It's the inevitable consequence of a missing library— a library so obvious in hindsight that the absence is almost embarrassing. Apache DataSketches gave the world sketches but never wired them to a query optimizer. Iceberg's Puffin specification gave us a sidecar format but no producer/consumer library to fill it. AQO, the PostgreSQL-only adaptive estimator, has feedback but no portability between engines. The three pieces have been sitting on three different shelves for years, and nobody has bolted them together.

I shipped samkhya v1.0.0 today to bolt them together. And to do four other things while I was at it.

Repository: github.com/singhpratech/samkhya. Technical paper: /publications/samkhya-portable…. This is the engineering tour.

The five-layer stack, in plain English

samkhya is सांख्य— Sanskrit for "enumeration, counting"— the name of the classical darshana whose entire discipline is counting reality's constituents honestly. The library has exactly that job. It is a 13-crate Cargo workspace under a single Apache-2.0 license with the explicit §3 patent grant, structured as five replaceable layers, each failing safely toward the engine's native plan when the layer above is missing.

Layer 1— portable stats. Five classical sketch families ship in v1.0: HyperLogLog (precisions 4–18, measured RSE 0.676% at p=14 / n=10⁶— comfortably below the Flajolet 2007 0.8125% envelope); Bloom filters via Kirsch-Mitzenmacher double hashing; Count-Min; equi-depth histograms; and a 2D correlated histogram that captures the pairwise column dependencies the four scalar sketches miss. Each sketch carries a stable KIND tag (samkhya.hll-v1, samkhya.bloom-v1) and a to_bytes / from_bytes serialization contract. A Puffin sidecar produced by the Python wheel is byte-identical to one produced by the DuckDB extension and fully readable by the DataFusion adapter. That byte-identity is the moat. No engine owns the stats. The sidecar does.

Layer 2— feedback recorder. The recorder hooks query execution at the adapter boundary, captures (plan template, estimated rows, actual rows) triples, and writes them to a SQLite sidecar keyed by template. A per-template residual model— a gradient-boosted tree under 100 KB on disk in the default backend— learns the systematic bias between what the planner thought and what actually happened, then surfaces the correction as a hint. This is the observe-and-hint pattern from Stillger's LEO (IBM, 2001), Marcus's Bao (SIGMOD 2021 Best Paper), and Anneser's AutoSteer (VLDB 2023)— the only learned-QO pattern with documented production deployment, full stop. Cold start sees the native plan. The recorder fires only when it has evidence.

Layer 3— LpJoinBound. The never-regress guarantee, made provable. This is the non-negotiable contract every other layer must honour. Every corrected estimate is bounded from above by a pessimistic ceiling derived from LpBound (Zhang et al., SIGMOD 2025 Best Paper)— LP relaxation over ℓp-norms of degree sequences, no machine learning involved. The samkhya refinement is strictly tighter than the Atserias-Grohe-Marx AGM bound (PODS 2008) on the star-5 join family at p=1. Not approximately tighter. Strictly tighter on every cell of the 30-cell evaluation grid. Wilcoxon signed-rank test gives W=0, p=1.73×10⁻⁶— a complete dominance result. Translated to wallclock: 40.95× speedup over native DataFusion 46 LpBound tightness, BCa 95% CI [30.93, 47.45]. A correction that breaches the LpJoinBound ceiling is rejected and the native estimate is used in its place. The worst case is silently degenerate. Never catastrophic. That is the whole point.

Layer 4— GPU batch inference (opt-in, via gpudb). Subplan enumeration is embarrassingly parallel: each candidate is an independent forward pass through a small GBT or PFN. CPU does this serially. The GPU collapses what would be a thousand-iteration loop into one CUDA or Apple Silicon Metal kernel launch. When samkhya is paired with my gpudb extension, the correction model scores thousands of subplan candidates in a single launch. Strictly opt-in: the default cargo build --release --workspace links no CUDA, no Metal, no GPU runtime of any kind.

Layer 5— the pluggable Corrector backend. This is the headline. One Rust trait (Corrector), four shipped backend slots:

GBT (default). Gradient-boosted-tree, sub-MB on disk, sub-millisecond inference, no external dependencies. The conservative classical bet, and what the cold-start safety analysis assumes.
TabPFN-2.5 (opt-in, tabpfn_http feature). Hollmann et al. ICLR 2023 + Prior Labs 2026— foundation tabular model. Measured P95 31.15 ms at batch size 8, sequence length 128, on RTX 4090 Laptop (BCa 95% CI [29.39, 35.32]); q-error reduction vs GBT on synthetic 7.84% (BCa 95% CI [2.21, 14.62], p=1.04×10⁻⁵).
LLM-via-HTTP (opt-in, llm_http feature). The viral one. samkhya-core::residual::llm::LlmHttpCorrector calls an HTTP server you control. Two reference servers ship in samkhya-gpudb/scripts/— llm_infer_server.py (Python FastAPI, canonical) and llm_infer_server.ts (Node TypeScript port, broader operator appeal). Documented backends: Anthropic Claude (claude-opus-4-7, claude-sonnet-4-6), OpenAI GPT-4o-mini, local Ollama (llama3.2:1b via http://127.0.0.1:11434). Plug your own foundation model in ~50 lines of glue. Wire contract is dead-simple: POST /infer {"features": [...], "baseline_estimate": <u64>} → {"estimate": <u64>}.
Dummy (transport-floor). Echo backend for measuring wire overhead independent of model latency. Produced the H1-A PASS— P95 0.07–0.11 ms across batch sizes 1/4/8/16/32, proving the LLM plug works at sub-millisecond cost when the model isn't the bottleneck.

The safety story is what makes this sane. Every backend— Claude, GPT, Ollama, GBT, TabPFN, anything you write tomorrow— is clamped from above by the LpJoinBound ceiling at Layer 3. If a hallucinating LLM returns "the join is 10¹² rows," LpJoinBound says "the provable ceiling is 4.2 million" and the planner sees 4.2 million. A miscalibrated TabPFN, a stale GBT, an LLM with a cosmic-ray bit-flip— none of them can break never-regress, by construction. That's the contract the field has been waiting for: let foundation models help with query planning, without giving them the keys.

What's measured, what's projected— being explicit. The transport-floor latency (0.07–0.11 ms P95) and the wire contract are MEASURED on the dummy backend (full receipts in bench-results/19_llm_corrector.md). The live-LLM end-to-end latency cells— Claude (~1.2s P95 paper-projection), GPT-4o-mini (comparable), Ollama (latency-bounded by local hardware)— are PROJECTED pending API-key budget and the 30-trial measurement campaign. The mechanism ships in v1.0; the headline live numbers are next-revision work. I would rather you know that now than discover it when you read the bench-results dossier.

The LpJoinBound clamp, visualised. Claude (amber), GPT-4o-mini (cyan), and local Ollama (emerald) each contribute a cardinality estimate; the crystalline lens— the provable pessimistic envelope— refracts and bounds every stream before it reaches the query-plan tree below. A hallucinating LLM cannot exceed the ceiling. The worst case is the engine's native plan.

Now the part nobody else in this field tells you

The standard playbook in learned cardinality estimation— for about eight years now, across roughly fifty papers— has been: hand-tune a system against a target workload, report the geometric mean of the wins in the headline, mention the losses in a §9 disclosures section that nobody reads. The credibility deficit that resulted from this playbook is exactly as large as the cumulative gap between those papers and the production deployments that followed them. Which is to say large. Naru is dead. NeuroCard is dead. MSCN is dead. DeepDB is dead. BayesCard is dead. None of them shipped into a production database. The 2021–2022 critique papers (Are We Ready For Learned CE?, In-depth Study of Learned CE) wrote the obituary collectively. The field has been quiet about why ever since.

Before the WAVE4-F head-to-head against native DataFusion 46 on the IMDb Join-Order-Benchmark Slow subset (n=55 paired warm-cache, scale factor 1), I pre-registered three upper-bound performance claims. ≥1.6× geometric mean on join-heavy queries. ≥1.35× on aggregate-heavy. ≥1.50× overall headline.

All three were falsified.

The measured geometric mean is 1.038× wallclock— BCa 95% CI [1.026, 1.056], Wilcoxon W=212 p=3.00×10⁻⁶. The Benjamini-Hochberg FDR procedure at q=0.05 rejects the null on 24 of 55 cells. The full record across the suite is 17 wins / 38 ties / 0 losses. Never-regress holds— that is the whole point of the LpJoinBound clamp— but the magnitude of the wins is small, and I am reporting it as such, in the headline section, where this sentence lives.

The TabPFN-2.5 backend had two pre-registered hypotheses. H1-A (P95 below 50 ms) passed comfortably. H1-B (≥15% q-error reduction over GBT on synthetic) failed on magnitude— measured reduction is 7.84%, statistically real, half the pre-registered effect size. The paper reports H1-B as falsified.

I am telling you this here, in the launch post, on the first day, in the second-most-prominent section, because the alternative is the §9-disclosures playbook that hollowed out this field's credibility in the first place. The cost of admitting a falsified pre-reg in public is smaller than the cost of having someone discover it on their own three months later. The 17/38/0 record is the appropriate evidence to weigh: a real, modest, statistically significant, never-regress improvement, in the only kind of public paired-warm-cache head-to-head whose results would actually deserve to be believed.

The interesting number in this release is not the JOB-Slow geomean. It is the 40.95× LpJoinBound tightness over AGM on the star-5 family. That is a theoretical-tightness result, with a wallclock translation on the synthetic topology, and it is the right way to communicate what this library actually contributes.

The 1000 → 42 demo (thirty seconds to feel the mechanism)

The repository ships a stats_propagation_demo example that proves the end-to-end path in plain Rust. A 1000-row table wrapped in DataFusion 46's default TableProvider reports num_rows = 1000 to the physical plan. Wrap the same provider with SamkhyaTableProvider plus the optimizer rule, and the physical plan reports num_rows = 42. The example prints, verbatim: "without rule: 1000, with rule: 42."

The DataFusion integration is intentionally a five-line change:

use datafusion::prelude::SessionContext;
use samkhya_datafusion::{SamkhyaTableProvider, SamkhyaOptimizerRule};

let ctx = SessionContext::new();
ctx.state().add_optimizer_rule(Arc::new(SamkhyaOptimizerRule::default()));
let provider = SamkhyaTableProvider::wrap(inner_provider).with_puffin_sidecar("orders.puffin")?;
ctx.register_table("orders", Arc::new(provider))?;

The samkhya_leaves_seen diagnostic on the optimizer rule confirms the corrected stats reached the physical plan. No fork of DataFusion required. DataFusion 46's Distribution framework already accepts external column statistics— samkhya simply supplies better ones.

What is in the v1.0 box, plainly

13 crates. Approximately 266 #[test] blocks across the workspace. 17 property tests. ~31 million cargo-fuzz executions, zero crashes. Criterion microbenchmarks for sketches and Puffin I/O. clippy -D warnings clean. The full workspace builds in under two minutes on a laptop with no network access. An ACM Artifact Evaluation v1.1 reviewer entry ships in REPRODUCIBILITY.md— Functional, Reusable, Available badges all in reach. Full reproduction budget for the published numbers: roughly 90 minutes wallclock on the reference hardware. Apache-2.0 single license with explicit §3 patent grant, matching DataFusion, Iceberg, Arrow, and ClickHouse— every downstream user gets the same patent grant, not a dual-license toggle.

Engine status, no hedge words:

DataFusion— three-layer integration (SamkhyaTableProvider + SamkhyaStatsExec + SamkhyaOptimizerRule). Production. First-class target.
DuckDB— Rust-client integration via the bundled feature, production. The cxx extension (cdylib + runtime LOAD samkhya;) ships as staticlib+rlib in v1.0; cdylib waits on upstream DuckDB Issue #11638.
Polars— Beta. Series-to-sketch helpers + lazy_collect_with_feedback behind the engine feature. Optimizer hook is upstream-blocked on Polars Issue #23345.
Postgres— Scaffold only. pgrx-shaped, double-gated behind pg_extension feature + samkhya_pgrx_enabled rustc cfg, pinned to PG17. Real planner/executor hooks land in v1.1 after pgrx ≥ 0.13.
Iceberg— Production. Puffin reader/writer with KIND-tag registration for all five sketch types.
Arrow— Production. IPC round-trip helpers, byte-identical serialization for all five sketch types.
gpudb— CPU fallback production. GPU and TabPFN-2.5 HTTP backends opt-in behind tabpfn_http.
Python— Single abi3-py39 wheel on PyPI as samkhya. Covers the portable-stats layer (sketches, Puffin reader/writer, ColumnStats). Use case: dbt-style nightly ELT writes the Puffin sidecar next to the Parquet file; the morning's DuckDB/DataFusion ad-hoc queries inherit it for free. No Python ML stack required.

Why this is the right shape now

The cardinality-estimation field has spent eight years producing extraordinary research and almost no embedded-tier production code. The pattern is older than the learned-CE wave. Stillger's LEO at IBM in 2001 was the first feedback-driven query optimizer to reach a mainstream server-class DBMS— and it was the last. Every learned system since has carried the same shape: server-class assumption about a long-lived optimizer process, model footprint that doesn't fit the embedded tier (40–300 MB), inference latency the embedded tier cannot afford (5–50 ms), and a cold-start story that handwaves around the worst case.

The 2021–2022 critique papers said this plainly. The production-database field then routed around it via adaptive query execution— a gorgeous technique that is structurally inapplicable to engines without a long-lived process to adapt within. The embedded tier— DuckDB, DataFusion, Polars, gpudb— has been waiting for a library that addresses its constraints, not the constraints of the systems whose obituaries those critique papers were writing.

samkhya v1.0 is my bet that the three pieces an embedded engine actually needs— portable sketches, feedback-driven residuals, a provable safety envelope— are independent enough to ship as one library, and that the pluggable model-backend slot above them is the right level of abstraction to admit whatever the field consolidates on next. GBT default is the conservative classical bet. TabPFN-2.5 opt-in is the bet that foundation tabular models are real and that the right way to use them is behind a pessimistic envelope rather than as a replacement for one. The LLM-via-HTTP backend is the bet that the third wave is already here— Claude, GPT-4o-mini, and Ollama are usable today, the wire contract is intentionally trivial so swapping providers is a 50-line change, and the safety envelope means the worst case is the engine's native plan. The library doesn't have to know what model you plugged in.

The honest measurement is the second bet: that the field's credibility deficit is repairable by pre-registration and falsification reporting, and that the right way for a sole-author project to participate in that repair is to do it first and conspicuously. Falsified pre-regs are information. The papers that hid theirs are the reason this field has the reputation it has.

What ships next, in the order I will close it

The v1.0 limitations are named explicitly in §9 of the technical paper. The v1.1 roadmap closes them in the order they were named:

Tighter effect-size attribution on JOB-Slow— cold-cache and Parquet methodologies, larger n, larger memory ceiling. The 1.038× geomean is partly methodology and partly intrinsic; v1.1 separates the two with a clean experimental design.
Broader workload coverage starting with STATS-CEB (Han et al., VLDB 2022). JOB-Slow is the field's standard but it is one schema. STATS-CEB is the obvious next workload.
In-process TabPFN-2.5 backend. The current HTTP round-trip eats 10–15 ms; collapsing it gets the P95 well under 20 ms.
Real Postgres planner/executor hooks. The scaffold is in v1.0. The implementation lands after pgrx ≥ 0.13.
Live-LLM end-to-end measurement campaign. The LLM-via-HTTP backend ships in v1.0 with a measured transport floor (P95 0.07–0.11 ms) and three documented providers (Claude, GPT-4o-mini, Ollama), but the 30-trial paired live-provider campaign hasn't run yet. ANTHROPIC_API_KEY and OPENAI_API_KEY need budget approval; Ollama just needs me to install it on the bench host. The mechanism is in v1.0; the measured live numbers are v1.1.

One sentence each, for the three audiences who will read this

If you have ever wanted to put Claude or GPT-4o-mini into the query path and stopped because nobody had built the safety envelope: that envelope is shipped. cargo add samkhya-core --features llm_http, point llm_infer_server.py at your favorite provider, and your DataFusion (or DuckDB, or Polars) optimizer will route every cardinality estimate through the LLM, clamped from above by a 2008-bound-tightening LP so a hallucination can never produce a worse plan than the engine's native one. That's the contract. That's the thing the field has been waiting for.

If you ship DataFusion, DuckDB, Polars, or Iceberg code today and you have ever cursed at a 1000-row estimate that should have been 42: cargo add samkhya-core and the five-line DataFusion snippet above gets you there in an hour. No LLM involved by default. The GBT backend is the conservative classical bet.

If you are a database researcher who has been quietly hoping someone would build the embedded-tier learned-CE library that fifty server-class papers couldn't: this is that library, the design is in the companion paper, the falsified pre-regs are in the headline section because that is where they belong, and the bench-results/ directory in the repository is where the receipts live— I would rather have the argument there than in the abstract.

The library's whole job is to count reality's constituents honestly. The release is doing the same thing about its own measurements. Both are deliberate. Both are how this field starts to look like the production field it always wanted to be.

samkhya v1.0.0— Apache-2.0. Repository: github.com/singhpratech/samkhya. Technical paper: /publications/samkhya-portable-feedback-driven-cardinality-correction-embedded-analytics. Companion to the prior gpudb release— the two share the embedded-tier engine target, the GPU-optional architecture, and the single-Apache-2.0 license posture.

On-Device AI Just Got Real

AI Explore — Sun, 28 Jun 2026 18:02:49 +0000

Apple's newest on-device model carries about 20 billion parameters, and on any given request it fires maybe one to four billion of them. That gap — 20B stored, roughly 3B running — is the whole story of 2026. The model that now ships inside the latest iPhone is no longer a shrunken, lobotomized cousin of the cloud model. It's a different kind of object: large in flash, small in motion, and it never phones home.

For three years the on-device pitch was mostly aspirational. Demos ran, latency was rough, quality trailed the API by a generation, and every serious AI feature still resolved to a per-token bill in someone's datacenter. In mid-2026 that stopped being true. Two releases — Apple's third-generation Foundation Models at WWDC on June 8, and Google's Gemma 4 family on April 2 — quietly moved the floor. Genuinely useful agents now run on hardware you already own, offline, for free.

The economics nobody priced in

Forget benchmarks for a second; the load-bearing fact here is accounting. When the model lives in the cloud, every inference is a metered event — input tokens, output tokens, a line item that scales linearly with usage and explodes the moment you wrap the model in an agent loop. Agentic workloads are the worst case for the token meter: a single "go do this task" can fan out into dozens of model calls as the agent plans, calls tools, retries, and re-reads its own output. The bill grows with your ambition.

Move the model onto the device and the marginal cost of an inference is approximately $0. No API key, no rate limit, no usage dashboard. You paid for the silicon once; every token after that is free in the only sense a product manager cares about — it doesn't show up on a monthly invoice that grows with your success. That single change rewrites which features are worth building. A background task that re-summarizes your inbox every five minutes is insane on a per-token plan and trivial on-device. So is an agent that quietly loops a hundred times to get one answer right.

And it isn't only cost. On-device means offline — the model works on a plane, in a tunnel, in a country where your cloud provider has no presence. And it means private in the literal architectural sense: the data never leaves the NAND. For a calendar, a photo library, a health log, or a half-written message, "these bytes physically did not transit a network" is a far stronger guarantee than any privacy policy.

Sparse beats big: the architecture that did it

The reason this works now isn't that someone discovered how to cram a frontier model into 3GB of RAM. It's that the model designs changed shape. The winning idea across both Apple and Google is the same: decouple how big the model is from how much of it actually runs on any given token.

Apple's AFM 3 on-device model uses what the company calls Instruction-Following Pruning (IFP). The full ~20B-parameter model lives in flash. For a given request, the system activates only the relevant ~1-4B parameters, swapping the needed "experts" into DRAM on demand. The phone never holds the whole model in working memory — it streams the slice it needs. That's how a 20B model fits inside a memory budget that physically cannot hold 20B of active weights.

Google's Gemma 4 attacks the same problem from two angles. The edge models — E2B and E4B — use "Per-Layer Embeddings" to keep the active footprint small: E4B carries roughly 8B total parameters but runs with about 4.5B effective. Its bigger sibling, a 26B mixture-of-experts, only lights up a fraction of its experts per token. MoE and per-layer tricks are Apple's IFP insight wearing different clothes — most of a large model is dead weight on any single token, so don't pay to run it.

The hardware finally met the software halfway. The neural accelerators (NPUs) now standard in phones and laptops run 4-8B-class models at genuinely usable speeds. The practical question shifted from "can it run at all" to "which model fits this RAM tier" — and that's a routine product decision, not a research problem. Google says the Gemma 4 edge models run "completely offline with near-zero latency" not just on phones but on a Raspberry Pi and an NVIDIA Jetson Orin Nano; the prior generation's E4B reportedly fit in about 3GB of RAM.

These are not toy models anymore

The capability jump is real, and it's broadest where it matters for everyday use: multimodality. AFM 3's on-device model is now multimodal — it takes images in, and Apple reports human raters preferred its image understanding about 61% of the time over the previous generation. Its on-device text-to-speech scored 4.24 on a 5-point mean-opinion scale versus 3.82 for the baseline — roughly the difference between "obviously a robot" and "fine, I'll actually listen to this." Gemma 4 ships native vision and audio, 128K context on the edge models, and 140+ languages.

The open-model leaderboard backs the claim up. Google's 31B dense Gemma 4 lands around #3 among open models and its 26B MoE around #6 on LMArena's text board — and Google's own framing, that these "outcompete models 20x their size," is the whole thesis in one line. The point of a small model in 2026 isn't to match GPT-class frontier reasoning. It's to be good enough at the 90% of tasks that don't need it, while running for free in your pocket.

What it still can't do

The honest caveat: the device model is not the frontier model, and pretending otherwise is how you ship a disappointing feature. Hard multi-step reasoning, long-horizon coding, deep research across large corpora — those still belong in the cloud, where a much larger model with a big context budget earns its token bill. Treat the small-model benchmark numbers that float around — figures in the mid-80s on MMLU for 14B-class models, high-60s for sub-4B ones — with suspicion; MMLU is saturated and gameable, and a leaderboard score tells you almost nothing about whether the thing can hold a five-step plan together. The right mental model is a hybrid: the device handles the fast, private, high-frequency work and hands off to the cloud only when a task genuinely outgrows it. The interesting engineering of the next year is the routing layer that decides which is which.

Apple opened the gates

The most underrated WWDC announcement wasn't the model — it was the door. Apple opened its Foundation Models framework to third-party and open models, with Swift packages for Anthropic's and Google's models on the way, and added agentic primitives plus on-device semantic search to the SDK. Translation: a developer can write an app against one local-first AI framework and let the device decide which model answers. That's the platform move. The model becomes a commodity inside it; the framework — the agent primitives, the semantic index over your own files, the routing — is the moat. Once the OS ships a free, private, capable model and a clean API to it, "add AI" stops meaning "add a cloud dependency and a billing relationship" and starts meaning "call a system function."

The take

The cloud-AI era trained everyone to assume intelligence is a utility you rent by the token. 2026 is the year that assumption cracked at the edge — not because device models got as smart as the frontier (they didn't), but because sparse architectures finally made "large but cheap to run" a real category, and the economics of $0 marginal inference are too good to ignore for the enormous class of features that never needed a genius in the first place. The cloud keeps the hardest problems. The device quietly takes everything else — offline, private, and off the meter. That's not a demo anymore. It's the new default, and most software hasn't been rewritten to assume it yet. The teams that rewrite first will look, briefly, like magicians.

The Coding-Agent Arms Race: Who Survives the H1-2026 Shakeout

AI Explore — Sun, 28 Jun 2026 18:02:46 +0000

TL;DR — Coding agents stopped being a checkbox in your IDE and turned into a four-way platform war in the first half of 2026. Anthropic is winning the model-and-product fight, OpenAI is winning distribution, and Cognition is winning the enterprise. The real moats are model cadence, install base, and price — not features. Pick your agent like you'd pick a vendor you might have to leave, because one of these companies is going to whipsaw your workflow before the year is out.

On June 9, 2026, Anthropic shipped two new models — Fable 5 and a restricted, higher-tier Mythos 5 — and stood up a new pricing class above Opus at $10/$50 per million tokens. The same day, OpenAI quietly added "Migrate to Codex" flows designed to import your Claude Code setup with a couple of clicks. Two of the most valuable companies on Earth, shipping on the same Tuesday, fighting over the exact same thing: the cursor in your terminal.

That is not a feature race anymore. That is a war for the developer's keyboard, and H1 2026 was the year it got bloody. If you're still picking a coding agent the way you'd pick a linter, you're underpricing the decision. The agent you wire into your workflow today is a bet on which platform survives — and several of them won't.

The numbers that ended the "feature" era

Follow the money and the stakes get obvious fast. Claude Code's annualized revenue reportedly went from roughly $1B in November 2025 to about $2.5B by February 2026 — a single product line, doing 2.5x in a quarter. Anthropic as a whole exited 2025 near $9B in run-rate; Dario Amodei confirmed a $19B run-rate at the Morgan Stanley TMT conference on March 4, 2026, and tech press pegged it near $30B by April — roughly 80x growth in a couple of years. The Series G — about $30B raised at a ~$380B post-money valuation in February — is the kind of number you only justify if you believe coding agents are infrastructure, not novelty.

Cognition, the company behind Devin, raised more than $1B at a $26B post-money valuation on May 27, 2026 — up roughly 2.5x in eight months from the $10.2B it carried in September 2025. Its ARR run-rate sits around $492M, with enterprise usage reportedly growing 50% month-over-month for six straight months and a customer list that includes Mercedes-Benz, NASA, and Goldman Sachs. When NASA is letting an autonomous agent touch code, the "is this a toy" conversation is over.

You don't see capital and revenue move like this around features. You see it move like this around platforms — the kind people get locked into.

The model train now leaves every six weeks

The single most underrated dynamic of 2026 is cadence. Anthropic alone shipped Opus 4.5 on November 24, 2025 — the first model over 80% on SWE-bench Verified at 80.9%, priced at $5/$25 per million tokens — then Opus 4.7 on April 16, Opus 4.8 on May 28 (which became the default within days and added a $10/$50 "fast mode"), and the Fable/Mythos drop on June 9. That's a new frontier roughly every six weeks.

OpenAI answered with GPT-5.5 on April 23, reportedly hitting around 88.7% on SWE-bench Verified and 82.7% on Terminal-Bench 2.0 at $5/$30, with a GPT-5.5 Pro tier at $30/$180. Google moved Jules onto a Gemini 3 Flash base on January 30 and onto Gemini 3.1 Pro for paid users by March 9. Three labs, all re-baselining their agents every few weeks.

Cadence is a moat the way a treadmill is a moat: it doesn't stop, and the cost of falling off compounds. A lab that ships a frontier model every six weeks can absorb a competitor's big launch in a month. A lab that ships twice a year cannot.

This is why I'd bet against any pure-play coding-agent startup that doesn't own a model. If your differentiation is the harness around someone else's weights, your roadmap is hostage to a release schedule you don't control — and your margins evaporate the moment the model underneath you gets cheaper or smarter without you.

Distribution is the moat nobody wants to admit

Here's the contrarian part: the best model is not guaranteed to win. The best distribution usually does, and that's where OpenAI is dangerous. Codex is no longer a CLI — it's a CLI plus macOS and Windows apps plus mobile, and on June 25, 2026, Codex Remote went GA, letting you drive a Mac or Windows host straight from the ChatGPT app on your phone. OpenAI is plugging an autonomous coding agent into the single largest consumer AI install base on the planet, then adding a $100/mo Codex Pro tier to monetize the power users. The Codex CLI itself is free, and by some reports it leads Terminal-Bench 2.1 at 83.4% against Claude Code's 78.9% and Gemini CLI's 70.7%.

Google's play is the same logic by another route: bundle. Jules went GA in August 2025, shipped a CLI and API in October, and now leans on Gemini subscriptions — free at 15 tasks/day, $19.99 AI Pro at 100/day, AI Ultra at 300/day. Google doesn't need Jules to be the best agent. It needs Jules to be the default one already sitting inside an account you pay for.

Anthropic's counter is that its product is genuinely ahead. Claude Code in 2026 added /code-review, an /ultrareview cloud bug-hunting fleet, "dynamic workflows" where Claude scripts dozens to hundreds of subagents on its own, scheduled cloud Routines, native binaries, and Artifacts. That's the most sophisticated agent surface shipping today. The open question is whether product depth beats a billion-user front door. History says it usually doesn't.

The Windsurf saga is the warning label

If you want a single story that captures why your agent choice is a strategic bet, it's Windsurf. In May 2025, OpenAI had a roughly $3B deal to acquire it. By July 11, 2025, the deal collapsed. Within days, Google paid about $2.4B to license Windsurf's tech and hire CEO Varun Mohan and his co-founder into DeepMind — a reverse-acquihire that pulled the brains out and left the body behind. On July 14, Cognition acquired what remained.

Then watch what happened to the product. Under Cognition it shipped Windsurf 2.0 on April 15, 2026, got rebranded to "Devin Desktop" on June 2, and its Cascade engine is being replaced by a Rust rewrite called "Devin Local" — reportedly about 30% more token-efficient — with Cascade reaching end-of-life on July 1, 2026. If you built your team's workflow on Windsurf in early 2025, you have since survived a failed acquisition, a brain drain, a new owner, a rebrand, and the sunsetting of your core engine — in under eighteen months.

That's not a freak event. That's the base rate for this market right now. Anyone building on a venture-backed agent should assume at least one ownership shock, one rename, and one engine swap inside their planning horizon.

Pricing whiplash is the new normal

The other thing that should scare you is how fast the rules change underneath you. Anthropic first imposed weekly usage limits on August 28, 2025 to loud backlash (it claimed under 5% of users were affected). Then in 2026 it reversed hard: 5-hour limits were reportedly doubled around May 6, and weekly limits were raised about 50%, effective through 6 PM PDT on July 13, 2026 — a move widely read as defensive against OpenAI's Codex push. Anthropic hasn't said whether the higher ceiling survives past that date. Plan your June around a limit that might revert in July.

On the other side, Cognition cut Devin's price from $500 to $20/mo with Devin 2.0 in April 2025, then on April 14, 2026 retired its Core and Team plans and pushed self-serve onto quota tiers — Free, $20 Pro, $200 Max, and Teams. (Worth noting: the current product is Devin 2.2, shipped February 24, 2026; there is no "Devin 3" — v3 is the API. Don't let a vendor's marketing math confuse you.)

When a market is consolidating and growth is the only metric that matters, price is a weapon, not a number on a page. Expect it to swing — up via rate limits, down via land-grab discounts — with very little notice.

So who's actually winning?

My ranking, stated plainly and happy to be wrong in three months:

Anthropic leads on model-plus-product. The cadence is unmatched, the agent surface is the deepest, and Claude Code's revenue curve is the most convincing single data point in the category. The risk is distribution — it has the best terminal agent and the smallest front door.
OpenAI leads on distribution and is closing the product gap. Codex Remote plus mobile plus a free CLI plus the "Migrate to Codex" funnel is a coordinated assault on Anthropic's installed base. If Codex's Terminal-Bench lead holds, this gets very close.
Cognition is winning the enterprise autonomy bet. A $26B valuation, $492M ARR, and NASA-grade logos say the "fire-and-forget agent" thesis is landing where budgets are biggest — even if Devin is narrower than a general assistant.
Google is the slow-moving giant that wins by default. Jules doesn't have to be first; it has to be already paid for inside Workspace and Gemini subscriptions. Never bet against bundling.

The companies I'd worry about are the harness-only startups with no model and no distribution. Windsurf showed how that movie ends.

What builders should actually do about lock-in

Stop optimizing for the best agent this quarter and start optimizing for the cheapest exit. Concretely:

Keep your workflow in portable formats. The most encouraging trend of H1 2026 is the move toward open interop — Devin Local now ships an Agent Client Protocol that lets third-party agents (Claude's agent, Codex, OpenCode) plug in, and MCP support is spreading across Jules, Claude Code, and the rest. Build your context, your tool definitions, and your review steps around MCP and protocol layers, not around one vendor's proprietary config. The agent should be swappable; your scaffolding shouldn't move when it is.

Treat the model as a commodity input. A frontier model every six weeks means whoever you favor today will be leapfrogged by lunchtime. Wire your stack so swapping the underlying model is a config change, not a migration.

Assume the price changes. Don't architect a process that only pencils out at a promotional rate. If your team's economics break when a $20 plan becomes a quota plan, or when a weekly limit reverts on July 13, you've built on sand.

Don't marry a single-product startup with your critical path. Use the best tool, absolutely — but keep a tested fallback warm. The cost of running two agents in parallel is trivial next to the cost of an unplanned engine sunset.

The closing take

The uncomfortable truth of this market is that there is no safe pick — only hedged ones. Anthropic has the best product and the cadence to defend it, but the smallest distribution. OpenAI has the front door and is sprinting to close the product gap. Cognition owns the enterprise but rides a narrower thesis. Google wins the people who never chose at all. Every one of them will change a price, a limit, or a product name on you before this year ends, and at least one well-funded name in this space won't make it to 2027 intact.

So pick the agent that's best for the work in front of you today — and build everything around it as if you'll have to leave. In an arms race, loyalty is a liability. Portability is the only real moat you actually control.