The 5.9GB LLM That Fits on a Phone — and the $280 Billion Question

A tiny lab out of Caltech just shipped something that made me stop scrolling. PrismML released Bonsai 2 27B, which takes Alibaba's Qwen3.8 27B — a solid, widely-used open model — and squeezes it down to 5.9GB. That's roughly a 9x to 10x cut in memory footprint. No cloud, no API key, no per-token bill. Just a model sitting on your PC, possibly on a high-end phone.

I've been burned by "compressed model" hype before. Most of it reads like magic tricks: shrink first, apologize for the quality later. So the number that actually matters here is 98%. Bonsai 2 matches 98% of Qwen's aggregate benchmark scores, up from 95% on the first Bonsai release back in March. That first model has been downloaded over 11 million times, which tells you the appetite for this stuff is real.

How the trick works

The short version: model weights normally need 16 bits each to store what the network learned. PrismML's approach, called "ternary" weights, collapses that down to three possible values — +1, -1, or 0. Fewer values to store, dramatically less space. It's the kind of idea that sounds obvious after someone explains it and impossible before they do.

To be fair, there's a ceiling here that even the founder admits. Compression will always cost something. Hassibi says 100% parity is "fairly academic" — and honestly, he's right. An uncompressed 27B model isn't perfectly accurate either, and benchmarks don't perfectly reflect real tasks. A 2% dip on a test suite rarely shows up as a worse conversation or worse code. The harness around the model matters more than people think.

The more interesting bet is what's next. PrismML says the next releases will target several-hundred-billion-parameter models, where there's "more room to compress without losing intelligence." If that holds, we're looking at frontier-ish capability running locally — free, private, offline. Ion Stoica (Databricks co-founder, Berkeley Sky Computing Lab) frames it simply: intelligence at your fingertips, running on hardware you already paid for, never shipped to a cloud.

There are also rumors of Apple talks. Both sides declined to comment, so file that under "would be huge, currently unconfirmed."

Meanwhile, the burn rate is real

The same day, the FT reported OpenAI expects to burn through $280 billion by 2030. Let that sink in — that's not a typo. Deeply negative cash flows as they pour money into infrastructure while facing price pressure on every front.

It's a strange moment for the industry. On one side you have a lab betting that intelligence can get dramatically cheaper to run. On the other, the biggest name in the game projecting a quarter-trillion-dollar hole. One of these trajectories is going to bend. My money's on the compression one — hardware that's already sold is the cheapest compute you'll ever buy, and PrismML's whole pitch is built on that.

China's compute answer

Huawei used its Connect conference to unveil the next-gen Ascend NPUs, and the Register's take is blunt: the 960DT could become China's best option, with performance "far exceeding anything the West could offer the Middle Kingdom." Nvidia can't sell its best stuff there, so Huawei's roadmap effectively sets the ceiling for Chinese AI training for the foreseeable future. The geopolitical angle aside, a credible non-Nvidia training stack is good for everyone who doesn't enjoy paying Nvidia's margins.

The dark side of on-device intelligence

Not all local AI is cute. Zimperium's researchers found a new Android trojan called RatHat that uses an AI assistant to navigate infected devices in real time. It serializes the accessibility tree to XML, sends it to a popular AI tool, and gets back navigation instructions — scroll here, tap there — to move around banking and crypto apps without a human operator in the loop. It enables wireless debugging, installs a Go-based agent that survives removal attempts, and even intercepts your uninstall confirmation to cancel it.

The AI part is the genuinely new wrinkle. Scripted malware breaks when the UI changes; an LLM-guided agent just adapts. Same trick that makes my coding assistant useful is what makes this thing slippery. The fix is boring and old-school: don't sideload APKs, don't hand Accessibility permissions to strangers, keep Play Protect on.

Self-improving Claude

One more from the pile: Anthropic says Claude now leads about 26% of the company's R&D — helping build the next version of itself. Cool in theory, slightly unsettling in practice, and the transparency push around "autonomous self-improvement" is worth paying attention to. I'd rather labs talk openly about this than discover it in a leak.

---

Quick add-on note: I've been running small local models for daily tasks — summarization, quick drafts, offline translation — and honestly, the gap between "runs on my laptop" and "API-only" keeps closing. Bonsai 2's 5.9GB footprint puts it in a genuinely useful range for a mid-tier PC. Whether it holds up on real workloads beyond benchmarks, I'll know after a few weeks of daily use. That's the test that matters.

If you're curious about the math behind everyday decisions, there's a neat little Math Calculator I keep bookmarked — handy when the numbers in a news cycle get as wild as $280 billion.