Running large models is expensive. Everyone in the industry knew this, but for a while it was someone else's problem — a future problem, once revenue caught up. In 2026, the bill has come due.
The phrase circulating now is "intelligence-per-token." Not capability in the abstract, but useful output per dollar of inference spend. It's an unglamorous metric, and that's kind of the point. After years of chasing benchmarks, labs are being forced to ask whether what they're building is actually economically viable to serve.
TurboQuant
Google's recent answer to this is TurboQuant, a compression algorithm built specifically for long-context inference. Feeding a model 100K+ token prompts — the kind of input needed for serious document analysis — has always been memory-intensive. At scale, serving those requests gets expensive fast.
Quantization itself isn't new. Reducing the numerical precision of model weights to cut memory and compute overhead has been standard practice for a while. What Google appears to have done differently with TurboQuant is apply compression directly at the attention layer, which is where memory usage spikes during extended context processing. That's a targeted fix for a specific bottleneck, which is more interesting than broad quantization schemes.
Whether it holds up in production at the margins they're claiming is a different question. But directionally, it's the right problem to be solving.
Sora
The harder story is Sora. OpenAI reportedly pulled the video generation tool in March 2026, with compute costs running somewhere around $15 million a day and revenue not close to covering it. For a product that launched with genuine excitement, that's a difficult number to sustain.
Video generation is just expensive in a way that text isn't. Each second of output requires a lot of compute at inference time, and the efficiency gains that make text models increasingly cheap to serve don't translate cleanly to video. You can compress, you can distill, but at some point you're still moving enormous amounts of data to generate a few seconds of footage.
Sora's exit has unsettled the broader video-gen space. Runway, Pika, and others are watching. The question no one wants to say out loud is whether consumer video generation is actually a viable product at current compute costs, or whether it only works if someone is willing to absorb years of losses waiting for hardware to catch up.
Where This Leaves Things
TurboQuant and Sora's shutdown are two responses to the same underlying pressure. One bets that smarter compression can make expensive models affordable to serve. The other suggests that when compression alone isn't enough, you cut the product.
What this likely accelerates is investment in smaller, specialized models — not because they're more impressive, but because they're cheaper to run and easier to build a business around. The capability conversation isn't going away. But for the first time in a while, it's sharing space with a much more boring question: can you serve this at a price that makes sense?
Top comments (0)