OpenAI's Jalapeño Chip Beats Nvidia on Efficiency

That "104x better than Nvidia" number is real. It's also the most carefully constructed version of the truth OpenAI could have published. Both of those things are simultaneously true, and understanding why tells you more about where AI infrastructure is heading than any single data point does.

Let's start with what Jalapeño actually is. It's an ASIC—application-specific integrated circuit—built in partnership with Broadcom, and it's not a training chip. As The Verge reports, it's purpose-built for inference: the phase where a model that's already been trained actually does something—answers your prompt, runs an agent step, generates tokens. The traditional tension in inference hardware is between latency (how fast you get one answer) and throughput (how many answers you can serve per second cheaply). OpenAI's VP of hardware Richard Ho framed Jalapeño's core claim as not making you choose between the two.

That's the pitch. Here's the measurement.

The Benchmark Math, Unwrapped

OpenAI used InferenceX, a public benchmark from SemiAnalysis that tries to capture the full lifecycle of serving a request rather than synthetic matrix multiply scores. The comparison baseline was Nvidia's best result on that same benchmark at the time of testing—GB200 or GB300, depending on the model. Three models were tested: GPT-OSS 120B, DeepSeek R1 at 670 billion parameters, and Kimi K2.5 at one trillion parameters. Notably, none of these are OpenAI's own frontier models; they're all public weights, two from Chinese labs, so results are at least theoretically reproducible.

Here's the number that matters for interpreting everything else: OpenAI normalized results by power, not per chip. Jalapeño is rated at 700 watts. The GB200 is rated at 1,200. The GB300 at 1,400. So before you see a single performance figure, Jalapeño is already working with roughly half the power budget of what it's being compared against. And according to Quartz, Jalapeño's measured sustained draw stayed at or below 550 watts during these workloads—meaning they normalized against a rated figure more generous than what the chip actually pulled.

With that lens on, the real-world numbers are still genuinely good. On peak throughput across the three models, Jalapeño delivered 1.5x to 1.9x more AI work per watt. On latency, the advantage is more pronounced: DeepSeek R1 full-request latency came in at 1.65 seconds on Jalapeño versus 5.99 seconds on the Nvidia system—a 3.6x gap. Kimi K2.5 ran at 1.56 seconds versus 5.31.

And then there's the 104x number. OpenAI ran a scenario where they pinned Jalapeño to the Nvidia system's fastest decoding speed and measured how much throughput each could serve at that constraint. On DeepSeek R1, that pinning point was 169 tokens per second per user. At that speed, Jalapeño served 12,258 mixed tokens per second per kilowatt. The Nvidia system managed 118. That's 104.3x.

What's actually happening there: you're forcing the Nvidia hardware to run at the absolute red line of its interactivity range, where its efficiency has tanked, while Jalapeño is still operating comfortably within its normal band. The video's AI Revolution channel puts it plainly—it's "the most flattering possible framing." The intermediate points on OpenAI's own chart tell the real story: the ratio climbs from 1.9x at peak efficiency, through 6.6x, 9.1x, 17.8x, and then rockets off as you push toward that extreme. The honest claim, which is also a real and significant one, is that Jalapeño sits on the outer edge of the throughput-versus-latency Pareto frontier across the full tested range. That's not nothing. That's actually quite a lot.

Why the Architecture Is the More Interesting Story

Language model inference splits into two phases with different performance bottlenecks. The prefill phase—processing your prompt—is compute-bound. The decode phase—generating tokens one by one—is bound by memory bandwidth. Most chips that excel at one tend to give back gains waiting on the other, plus communication overhead as data shuffles between cores.

Jalapeño was designed specifically to minimize that movement. Model state, including the KV cache (the memory structure that lets a model track context), can be explicitly placed and kept local. The network fabric is treated as a core architectural element rather than an afterthought. Ho's team describes the result as "fungible"—it can flex between prefill-heavy and decode-heavy workloads as the mix shifts, which is exactly the demand pattern that agentic systems produce.

What's arguably the most underreported part of the whole announcement: AI helped design a meaningful chunk of this chip. The team went from initial design to tape-out in nine months, and OpenAI credits models with exploring implementations, shortening verification loops, and optimizing arithmetic circuits. AI-generated kernels for selected attention and mixture-of-experts blocks in GPT-OSS ran 1.5x to 1.8x faster than what human experts had written—for those specific blocks, not the full model, and OpenAI is careful to flag that distinction. Still: a chip with AI in its design lineage, running kernels written by AI, serving AI workloads. The loop is getting shorter.

Deployment-wise, small volumes ship by end of 2026, ramping through 2027. Generation 2 is in deep development; Gen 3 is taking shape. And Ho was explicit that this isn't a pivot away from Nvidia—OpenAI's compute plan still leans heavily on what he called "very good partners," Nvidia included, for both training and inference. Jalapeño is an addition to the stack, not a replacement. Worth keeping in mind when you see the GB300's real-world performance under production agentic load—Nvidia's hardware is doing serious work; the question is whether purpose-built inference silicon can serve that specific workload class more cheaply.

Elsewhere in the AI Week That Wouldn't Slow Down

While OpenAI was publishing benchmark charts, Anthropic was running a much quieter operation. Two model codenames—Melon and Marshmallow—briefly surfaced in developer communities before getting yanked offline within hours. A leaker who got hands on Claude Marshmallow-EAP described Melon as reading like a Fable-tier checkpoint and noted that Anthropic appears to be investing heavily in 3D spatial reasoning via reinforcement learning. The rumor circulating is that Fable 5.1 is already in a gradual rollout to some users, possibly alongside Sonnet 5.1. Anthropic has not officially confirmed that either codename maps to Fable 5.1, and the test prompt circulating to detect whether you're in the rollout has some circular logic baked into it. Take the speculation for what it is.

What's less speculative: Anthropic's own team has acknowledged that Opus 5 is having a moment. The person who runs Claude Code at Anthropic called the model "very stubborn right now," describing it as apologizing in loops instead of fixing errors when corrected, and said getting it back to feeling "consistent and warm" is the top priority. The secret-nerf theory that spread through developer communities got shot down—the parameter changes spotted were a number mapping issue in an internal test API config, with internal evals showing no regression. Which doesn't mean users aren't experiencing something real; it means the cause isn't what people thought.

Meanwhile, Alibaba shipped Qwen3.8-Flash with the cost story doing most of the work. According to Reuters, compared to Qwen3.7-Plus it cost a fraction of the training budget, comes multimodal with stronger coding and office task capabilities, carries a 262,144-token default context window expandable to one million, and is priced at roughly 15 cents per million input tokens. Alibaba also open-sourced weights for Qwen3.8-Flash-Next, which it describes as a prototype of the architecture headed into the Qwen 4 family—letting the open-source community poke at next-gen design choices before the flagship ships. The strategic context: e-commerce growth has stalled and AI is now Alibaba's primary revenue driver, backed by a heavily discounted 80 billion Hong Kong dollar share sale announced over the weekend.

The week's most quietly useful development was Anthropic merging the memory systems behind Claude Chat and Claude Cowork. Anything Claude learns in one surface now carries into the other—the project context you built over weeks in chat doesn't evaporate when you hand work to Cowork. Memory builds by topic mid-conversation rather than only summarizing at the end. The full memory store is readable, editable, and deletable by users. Sensitive categories—health data, race, ethnicity, religion, politics, gender identity—are off by default, with toggles and save notifications. Some categories won't be stored regardless of settings: government IDs, social security numbers, criminal history, immigration status.

---

The Jalapeño numbers are real and the architecture story is genuinely interesting. The 104x figure is also a benchmark constructed to maximize a particular comparison at a particular operating point. Both remain true. What the week collectively suggests is that the infrastructure layer of AI—the chips, the memory systems, the cost curves—is moving as fast as the models sitting on top of it, and the companies building both simultaneously are starting to have a different kind of conversation with their hardware suppliers than they did two years ago.

— Yuki Okonkwo, AI & Machine Learning Correspondent