GPUs are expensive, power-hungry, and, when running complex AI agents, notoriously lazy. They spend more time waiting than working. The assumption has been that this inefficiency is a fundamental hardware flaw, a problem you throw more money and silicon at. French startup Kog is betting that assumption is wrong, and their approach of "deep-level" software optimization just attracted 200 tangible business leads on the strength of a single tech demo, according to TechCrunch.
The bottleneck isn't the GPU's sheer compute power. It's the traffic jam inside it. Every time an AI workflow needs to fetch data, switch tasks, or handle a branching decision, the mighty GPU cores idle. For businesses deploying agentic systems that chain multiple reasoning steps, these delays compound into hours of wait time and massive, unused infrastructure costs.
Kog's pitch is seductively simple: unlock the latent performance already sitting in your existing AMD MI300X or NVIDIA H200 datacenter GPUs. Why buy a new, specialized inference chip like Cerebras's if you can get a 10x or 30x speedup with software? The promise is an immediate, substantial upgrade to the most expensive line item on an AI project's balance sheet.
Why AI Efficiency Is Now a Business-Critical Feature
The narrative around accelerating AI has focused on bigger, faster, more specialized chips. But the economics have shifted. The real pain point isn't training colossal models anymore, it's the operational cost of using them repeatedly in complex, real-world workflows.
Take a developer using Claude Code. They might wait hours for a result. That's a direct productivity tax, which is why Anthropic's Fast Mode commands a premium price. For startups where AI-generated code or no-app app builders are core to the product, speed isn't just nice, it's revenue. A slower model means fewer iterative cycles per day and a worse user experience. This is the market Kog is targeting: professionals and companies for whom speed is a competitive lever, and who are already sitting on underutilized GPU hardware.
This mirrors a broader trend we see in startup efficiency. In a resource-constrained environment, getting more from what you have is paramount. It’s why founders are moving beyond generic pitch decks scrapped by investors in under 3 minutes to data-driven narratives. The same rigor is now being applied to the AI stack itself.
Digging Into the GPU's Ignored Backstage
Everyone focuses on the AI model, the chef in the kitchen. Kog focuses on everything else: the layout, the pantry runners, the timing of the dishwashers. Their method involves a deep, sometimes grunt-level analysis of GPU kernel execution and memory movement.
“There’s this mindset of understanding the laws of physics, and the laws of the GPU in order to make the most of them,” says CEO Gaël Delalleau.
Delalleau's background is telling. He studied solid-state physics and was an offensive cybersecurity researcher, a four-time finalist at DEFCON's CTF tournament.
This combination is Kog's secret sauce. The physics mindset pushes them to understand the hardware's fundamental constraints. The hacking mindset pushes them to reverse-engineer things at a very low level...to understand how it works, and to try to use it to achieve a goal for which it wasn’t necessarily designed.
Their process is to profile a workflow with near-microscopic precision, find where milliseconds are lost, often in data fetches or context switches, and rewrite the GPU's instruction flow to eliminate those stalls. It's not about changing the model's architecture. It's about orchestrating the work so the GPU's compute engines are never starved for a task.
From Impressive Demo to Giant Leap
Kog's May tech preview made waves on Hacker News by delivering 3,000 tokens per second on a single request. That's staggering speed. But the fine print is crucial: they achieved this with a purpose-built, small 2-billion parameter model called Laneformer 2B, which they have since open-sourced.
This is the crux of both their promise and their risk. Delalleau is confident the same "deep-level" optimization principles can scale to large language models with hundreds of billions of parameters. Skeptics point out that LLMs present fundamentally different, more complex memory and compute patterns.
For Kog, bridging this gap is an existential engineering challenge. "Since the launch, we've been fully focused on accelerating the development of larger models to meet the demand we've seen," Delalleau told TechCrunch. Their prospective customers weren't interested in fine-tuning small models; they want their existing LLMs to fly.
The company's stated milestone is to deliver 10x speed on a "major model" by September. Hitting that target is more than a technical win. It's the linchpin for their business. Delalleau stated that achievement will allow them to "start demonstrating customer traction and from there, raise our Series A."
The Fork in the European AI Road
Kog isn't operating in a vacuum. They are part of a small but noteworthy French push for sovereignty in the AI stack. They are backed by French Tech 2030 and Bpifrance, and supported by cloud provider Scaleway.
They also have a conceptual neighbor in compatriot startup ZML, which makes hardware-agnostic software that bypasses NVIDIA's CUDA. But Delalleau draws a different comparison: he sees Kog as more akin to Stanford's Hazy Research, drilling even deeper into GPU acceleration itself.
The choice Europe faces is whether to try to build its own AI chips from scratch, a monumental, capital-intensive task, or to pursue a software-driven strategy that maximizes the efficiency of existing (often American) hardware. Kog embodies the latter path. It’s a pragmatic approach that could let European companies and researchers compete without needing to win the fab race.
It’s a different kind of sovereignty play, focused on maximizing control and performance within the constraints of a global supply chain, a theme sometimes seen in Europe's complex regulatory approach to technology.
The New Economics of AI Product Development
If Kog’s deep-level optimization works as promised, it rewrites several rules for building AI products.
First, efficiency becomes a primary feature, not an afterthought. Developers will need to consider how their agentic workflows manage GPU resources from day one, because software gains can now rival hardware upgrades. The stack just got deeper; competitive advantage will come from understanding the layers beneath the model framework.
Second, the market for inference could fragment. The need for drastically faster, cheaper inference on existing hardware is proven, Kog's 200 leads confirm it. If they succeed, they create a new category: performance-tuning middleware for GPUs. This challenges the narrative that you must move to specialized AI silicon for high-performance inference.
Finally, the economics of AI features change. Making a model 10x faster at the same cost doesn't just mean faster answers. It means you can afford to run more complex, multi-step agentic logic. It makes sophisticated AI accessible to companies that can't afford to carpet their data centers with the latest H100s. The value shifts from just buying more compute to investing in smarter software that fully utilizes the compute you already own.
The bottom line: Kog’s story is a test case. It asks whether the most significant near-term gains in AI will come from silicon or from code. Their September target for a 10x speedup on a major LLM is the deadline for an initial answer. If they hit it, they won't just secure their Series A. They'll prove that one of the most expensive problems in AI today might be solved not with a new chip, but with a deeper look at the ones we already have.
Why This Changes Everything
- Kog's software can potentially deliver a 10x to 30x speedup on existing datacenter GPUs, dramatically cutting the operational cost of deploying AI agents.
- It challenges the prevailing 'bigger chip' narrative by focusing on unlocking latent hardware performance, which could reshape procurement strategies.
- By reducing GPU idle time for complex, multi-step AI workflows, it directly improves productivity and revenue potential for businesses reliant on AI services.
Originally published on XOOMAR. For more news and analysis, visit XOOMAR.
Top comments (0)