The headline everyone's running with
OpenAI just published benchmark results for Jalapeño, its first custom inference chip, built with Broadcom. The numbers, on paper, are absurd:
- 1.5x–1.9x more throughput per watt than Nvidia's GB200/GB300 (Blackwell) systems
- 1.7x–3.6x lower end-to-end latency
- 2.1x–4.1x higher performance on interactive workloads
- GPT-OSS 120B hit 85,448 mixed tokens/sec per kilowatt; DeepSeek R1 hit 19,641; Kimi K2.5 hit 18,195
- On DeepSeek R1 at low concurrency, Jalapeño pushed ~700 tokens/sec per user — without speculative decoding, the trick everyone else leans on to hit numbers like that
Rated at 700W, it ran the tested workloads sustained at or below 550W. Design started mid-2024, taped out November 2025 — a 9-month design-to-fab cycle, ~16 months total. For a first-generation chip built by a company whose day job is not silicon, that's genuinely fast.
Dylan Patel, CEO of SemiAnalysis, whose InferenceX suite ran the benchmark: "Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin."
Nvidia earnings just dropped. The timing is not an accident. This is a press release wearing a lab coat.
Now read what SemiAnalysis actually wrote
Buried under the headline numbers is the part nobody's screenshotting:
1. The comparison target is wrong. Jalapeño uses HBM4 memory. Blackwell (GB200/GB300) uses HBM3E. That's not a fair fight — it's comparing a chip against last generation's memory bus. The apples-to-apples competitor is Nvidia's Vera Rubin, which also ships on HBM4. SemiAnalysis' own numbers: Rubin gets roughly 5.4x the performance-per-megawatt of GB200. Jalapeño's margin against Blackwell mostly evaporates the moment you swap in the chip that's actually its peer.
2. It's a spec-sheet fight, not a production fight. Jalapeño (B0 stepping): 13.4 PFLOPs at 700W TDP. Rubin: 17.5 PFLOPs at 900–1,150W. Rubin is faster in absolute terms and hasn't even been benchmarked against Jalapeño yet — because Jalapeño hasn't been tested against it. OpenAI picked the fight it could win.
3. The workload was easy mode. All of this ran on 8k-token-in, 1k-token-out contexts. That's a toy compared to what production inference actually looks like — long context windows, multi-turn conversations, agentic tool-calling loops. SemiAnalysis flagged this directly: their AgentX benchmark, built to stress exactly those realistic patterns, "hasn't been run on Jalapeño hardware yet." You don't get to claim you beat the incumbent until you've run the workload the incumbent actually serves in production.
4. OpenAI supplied the numbers. SemiAnalysis' own disclosure: "All numbers are provided to us by OpenAI. We verified the InferenceX runs in person in the lab" — but they did not run the full independent suite themselves. That's a meaningfully weaker claim than "we benchmarked this ourselves and here's what we found."
5. It's not shipping. These are engineering samples. Small prototype deployments land end of 2026, "very small volumes." Real production ramp is 2027. Full-scale production isn't projected until 1H 2028. Larger models — DeepSeek V4 Pro, Kimi K3 — haven't touched this hardware at all.
None of this makes Jalapeño fake. A 16-month-old chip from a company that has never built silicon before, landing in the same benchmark neighborhood as Nvidia's current flagship, is a real engineering result. But "beats Blackwell" and "is competitive with Blackwell on a curated, easy benchmark, against the wrong generation, on numbers we didn't independently verify, on hardware that won't ship at scale for two more years" are very different sentences. Only one of them fits in a tweet.
Why you should still care
Strip away the marketing and there's a real signal here: OpenAI just proved the vertical-integration playbook works on a 16-month clock. Google did it with TPUs over a decade. Amazon did it with Trainium/Inferentia over several generations. OpenAI apparently did a credible first pass in a year and a half, with a manufacturing partner (Broadcom) instead of in-house fab expertise.
If you're building anything that burns serious inference budget — RAG pipelines, agents, anything running Llama/DeepSeek/Kimi-scale open models at volume — the thing to track isn't "did Jalapeño beat Blackwell." It's tokens-per-dollar at your actual context length and concurrency, not the vendor's context length and concurrency. Every chip vendor, OpenAI included, will hand you the benchmark that flatters them. Your load isn't 8k-in/1k-out at low concurrency, and it isn't running the model they picked.
Nvidia isn't sweating a single generation of custom silicon. CUDA lock-in and the software ecosystem are still the moat — a faster chip with a worse toolchain loses to a slower chip with a mature one, every time, in production. What Nvidia should be sweating is that this is now a repeatable pattern: OpenAI, Google, Amazon, and Meta are all fielding or fielding-again their own inference silicon, and every one of them is a customer Nvidia used to have exclusively.
The takeaway for engineers, not investors
If a vendor — any vendor, including the one that just published this — hands you a benchmark, ask three questions before you cite it in a deck:
- Who ran it, and did they run it, or did the vendor run it and hand them the CSV?
- Does the workload match your production shape (context length, concurrency, model), or is it the shape that makes the new hardware look best?
- Is this shipping, or is this a demo with a shipping date attached?
Jalapeño passes the engineering bar. It does not yet pass the "you can build on this" bar. Watch AgentX results and the 2027 production numbers — that's when this story gets real. Until then, it's a very good demo with excellent PR timing.
Sources: OpenAI's Jalapeño first results, SemiAnalysis technical analysis, the-decoder.com benchmark breakdown, CNBC coverage.
Top comments (0)