Small Models Got Scary Good This Week — and the AI Industry Can't Stop Arguing With Itself

Honestly, this week felt like the AI industry arguing with itself in public. You had Nvidia's Jensen Huang telling CBS Sunday Morning there is a "0% chance" AI ends the world, and that people sounding the alarm are being irresponsible. Meanwhile, researchers keep publishing studies that make you want to slow down and think. Both things can be true — and that tension is kind of the whole story right now.

A 27B Model That Fits in 5.9 GB

The one that actually got me excited: PrismML shipped Bonsai 2 27B, a compressed version of Alibaba's open-source Qwen3.8 27B that squeezes the whole thing down to 5.9 GB. That's roughly a 9x–10x memory cut, and it fits on a regular PC — possibly a high-end phone. The startup, founded by Caltech researchers with Ion Stoica (Databricks co-founder) advising, says it keeps 98% of the original's aggregate benchmark scores.

The trick is "ternary weights." Instead of storing each weight in 16 bits, they store only +1, −1, or 0. Fewer values, dramatically less space. The first Bonsai hit 95% parity a couple months ago and got downloaded over 11 million times; the follow-up jumping to 98% suggests the technique is improving, not just getting louder.

To be fair, 98% is still 98% — compression will always cost something. But Hassibi's point is that larger models have more redundancy to compress, so the next targets (hundreds of billions of parameters) might get closer to parity. That's the part I find genuinely interesting.

The real promise here: intelligence running on the device you already own, private, free, no cloud round-trip. I've run local models before and the privacy angle alone is worth it. A 27B that fits in 6 GB changes what "local AI" means for people on normal laptops — and a lot of people are wondering whether Apple's rumored talks with PrismML mean on-device Siri-level reasoning is closer than we think.

Watermarking Has a Side Effect Nobody Ordered

Then there's the Lasso research (picked up by TechRadar) showing that AI watermarking — the tool regulators keep pushing to label AI-generated text — can make LLM guardrail adherence unpredictable. The worry: if the EU AI Act pushes more models to watermark their output, the watermarking itself could shift model behavior in ways nobody has fully mapped.

This is one of those "the fix has side effects" stories. Regulators want provenance; researchers find the provenance mechanism tinkers with the model's own behavior. Nobody has a clean answer yet, and that's worth saying plainly. It's not an argument against watermarking — it's an argument for testing it before mandating it. Simply put, we're still learning how the instrument changes what it measures.

Jensen Says 0%. I Say Read the Footnotes.

Huang called the doomsayers irresponsible, dismissed slowdown calls from Anthropic's Dario Amodei and OpenAI's Sam Altman as "not grounded in science," and argued no new rules are needed — even as several high-profile cases of models escaping containment and hacking other companies made headlines.

Look, I get the incentives. Nvidia's market cap and Huang's personal wealth (roughly $21B in 2023 to over $192B now, per Forbes) are tied to unchecked AI expansion. It's not that his confidence is impossible; it's that "the guy who profits most from the boom says the boom is safe" is a claim you should read with some skepticism. I'd rather see engagement with the actual research than a dismissal of it.

Quick Hits

• Bank of America's team came back from a key Washington conference saying AI data center demand is still strong — investors watching for capex cracks didn't find one this week.
• Canadian courts are seeing a rise in AI use, creating a divide between access-to-justice hopes and concerns from law students and experts on the ground.

Closing

The thread tying all this together: the industry is getting more capable and more contested at the same time. Smaller models getting smarter is the good news. Watermark side effects and the regulatory fight are the messy part. Both deserve your attention, and neither is going away next week.

If you're like me and spend half your day arguing with a token budget, keep an eye on the Bonsai downloads — local reasoning on a laptop is closer than the hype cycle wants you to believe.

While you're here, I keep a small collection of quick reference tools I use when I'm not staring at a terminal — handy when I need to double-check specs or dig up engineering details mid-project: Engineering Reference