My GPU database was 110 slower than its own kernel. I published the benchmark anyway.

TL;DR — A reviewer asked what my GPU DuckDB extension does to whole queries, not just kernels. I measured it honestly: native DuckDB won every shape, 3×–109×. I published that table in my own release notes. Writing it down forced the structural diagnosis, and the very next release closed every cell to within 0–20% of native — by removing the GPU from the SQL hot path. On unified memory, that was the fix.

I maintain gpudb, a DuckDB extension that runs SQL aggregates on NVIDIA CUDA and Apple Silicon Metal. Its kernel-level numbers are genuinely good — the fused multi-aggregate Metal kernel does SUM+MIN+MAX+COUNT over a TPC-H SF10 column in 1.13 ms, ~25× faster than DuckDB's 16-thread CPU path.

Then, during review of my DuckDB community-extensions submission, someone asked the question every GPU project dreads:

Your scorecard measures the operator on resident data. What does a user actually see when they run a whole query through the full pipeline?

The benchmark I didn't want to run

Rewritten TPC-H Q6 and Q1 plus a high-cardinality GROUP BY, through the stock DuckDB CLI v1.5.2, SET threads=16, warm cache, median of 5 runs, correctness gated before timing (the Q6 total matches the published TPC-H reference answer). Both sides cast ::DOUBLE identically.

Native won everything:

Query shape	native	gpu_sum (v0.2.0)	native advantage
Q6 · SF1	0.002 s	0.006 s	~3×
Q6 · SF10	0.017 s	0.057 s	~3.4×
Q1 · SF1	0.011 s	0.33 s	~30×
Q1 · SF10	0.098 s	3.45 s	~35×
GROUP BY l_orderkey · SF1	0.010 s	0.944 s	~86×
GROUP BY l_orderkey · SF10	0.092 s	11.05 s	~109×

I put that table in BENCHMARK.md and the v0.2.0 release notes under the heading "honest: native CPU wins," with the conclusion stated plainly: in v0.2.0, gpu_sum is not a way to make whole queries faster.

Where 109× actually came from

The punchline that makes this a story instead of just a failure: the SF10 GROUP BY that loses 109× end-to-end and the operator benchmark where the same kernel wins 1.30× are the same data and the same kernel. The ~140× swing was entirely the extension's execution path.

Writing the honest section forced the diagnosis. Four causes, in order of damage:

1. Buffering, twice. DuckDB drives aggregates through per-chunk update callbacks. The extension appended every value into a BufferPool state — a full serialized copy of the column — before the GPU saw a byte. Then finalize snapshotted that buffer into another vector. The copy itself was the cost, not the reduction.
2. A global mutex + per-state finalize. Every GPU dispatch serialized on one lock, and grouped aggregation finalized one group at a time — 15 million dispatch decisions at SF10, to sum ~4 values each.
3. DOUBLE on Metal never reached the GPU. Apple GPUs don't implement IEEE-754 doubles, so sum_f64 is a host-loop fallback. The Q6/Q1 "GPU" numbers were: full buffering overhead, zero GPU upside.
4. Q1's 4-group regime is a structural CPU win. A four-entry hash table lives in L1. No accelerator fixes that.

Note what this list amounts to: the loss was structural, not a tuning problem. That prediction went into the release notes too.

The fix was deleting the GPU

v0.3.0 rewrote the aggregate path from "buffer every value, reduce at finalize" to streaming running accumulators — the same algorithmic shape as a native DuckDB aggregate:

v0.2.0: chunk → append into BufferPool (full copy) → finalize: snapshot → mutex → per-state reduce ×15M
v0.3.0: chunk → 24-byte accumulator += → finalize: return the value

The BufferPool, the global mutex, and the batched GPU finalize are gone. The state is a 24-byte POD. And yes — the SQL aggregate path in a GPU database extension no longer dispatches to the GPU, and that's why it got ~100× faster.

On unified memory (Apple Silicon), CPU and GPU read the same LPDDR5X at the same bandwidth. For a single streaming pass like SUM, shipping the column to the GPU buys nothing and costs everything. The GPU earns its keep where there's real parallel work per byte — fused multi-aggregates, operator-level grouping, and joins — not in a per-value accumulate behind a callback interface.

One release later

Same suite, same machine, same methodology:

Query shape	native	v0.3.0	ratio	was (v0.2.0)
Q6 · SF1	0.002 s	0.002 s	1.00×	~3×
Q6 · SF10	0.017 s	0.018 s	1.06×	~3.4×
Q1 · SF1	0.011 s	0.012 s	1.09×	~30×
Q1 · SF10	0.098 s	0.101 s	1.03×	~35×
GROUP BY · SF1	0.010 s	0.012 s	1.20×	~86×
GROUP BY · SF10	0.092 s	0.109 s	1.18×	~109×

The 11.05 s cell now runs in 0.109 s, with identical results. The honesty rules still apply, so the residuals are recorded, not hidden: the remaining 0–20% is unprofiled (defensive probes, callback indirection, native's fused specializations are the suspects), and the brand-new gpu_min(DOUBLE)/gpu_max(DOUBLE) overloads are ~1.5× slower than native — their value this release is existing at all.

Where the GPU actually goes: joins

Joins have real parallel work per byte — the profile where an accelerator outruns the memory system. A contributor (@lmangani) showed up with a real Metal hash join + on-device segment reduce (PR #43), verified at 9.9× on a 1M×10M inner join on M4 Max. That's the v0.4.0 headline.

Why you should publish your losses

The accounting on one embarrassing paragraph, about a day later:

• It produced the fix. Writing the honest section forced the diagnosis; the diagnosis predicted the rewrite; the rewrite closed the gap in one release. If I'd answered the reviewer with kernel numbers and moved on, the 11-second GROUP BY would still be shipping.
• It made the good numbers believable. A benchmark file that says "native wins 109×, here's why" earns the right to say "the fused kernel wins 25×" and be believed.
• It cost nothing. No ridicule, no exodus. What showed up instead: a better architecture, an external contributor with a 9.9× join, and this post.

The benchmark section where my project loses 110× has done more for the project than every section where it wins.

Everything is reproducible: repo · BENCHMARK.md · community-extensions PR #1898.