OpenAI's First Chip Outruns Blackwell in Early Tests
OpenAI showed up at the Hot Chips conference on Tuesday with something it hadn't offered before: actual numbers. Jalapeno, the inference processor announced last October and built alongside Broadcom, now has published benchmark results attached to its name.
What the tests measured
The chip was run on InferenceX, a benchmark maintained by the analysis firm Semianalysis. On the two figures that matter most to anyone serving a chatbot to millions of people, Jalapeno beat the best inference hardware you can buy today: tokens delivered per user, and throughput per kilowatt of power drawn.
Those two usually fight each other. Design for raw throughput and individual users wait longer. Design for snappy replies and you waste capacity. OpenAI says it moved both at once.
"The bottom line is that the results show a very, very significant performance advance over state of the art," Richard Ho, OpenAI's head of hardware, told reporters on a press call.
The comparison, and its asterisk
The benchmark ran against an Nvidia Blackwell system. That's a fair yardstick right now, but there's a catch worth stating plainly: Nvidia isn't standing still, and Jalapeno isn't shipping in volume yet.
Ho said the chip should reach deployment at the end of 2026 in what he called very small volumes, with anything substantial waiting until 2027. Whatever Nvidia has in the field by then won't be the hardware Jalapeno just beat. Early silicon benchmarks are a statement of intent more than a scoreboard.
Why build the chip at all
The score isn't really the story. The reason OpenAI thinks it can post one is.
The company designed Jalapeno in concert with the models meant to run on it, and says its own models pitched in on the development work. That position let it target specific stages of inference instead of general-purpose compute. Two stages get most of the attention: prefill, where the model reads your prompt before writing a word, and the communication phase, where data moves between parts of the system. Both, OpenAI says, tend to be where things stall.
"We designed Jalapeno to minimize data movement and communication delays," the company wrote in its blog post on the results. "This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase."
In plainer terms: rather than shuttling a conversation's working memory around the machine while the model writes back, Jalapeno pins it down and brings the right resources to it.
The longer bet
This isn't meant to be a one-off part. OpenAI has described Jalapeno as a multigenerational platform, with products, models, chips and memory designed together across future revisions.
Apple used that logic on phones. Google has used it on TPUs for years. It's also the reason an AI lab would accept the cost and risk of custom silicon instead of just buying more GPUs: when your workload is narrow, predictable and enormous, general-purpose hardware leaves efficiency sitting on the floor.
Why it matters
Throughput per kilowatt is the number to keep an eye on. Power, not chip supply, is increasingly what caps how much AI capacity a company can switch on, and a processor that serves more requests per watt raises that ceiling without anyone building a new substation.
If you use ChatGPT, none of this arrives as a feature. It arrives, eventually, as quicker answers and cheaper tokens. What OpenAI claimed this week is that it can deliver both from hardware it controls end to end. The claim gets tested in 2027, when Jalapeno has to hold up at scale against whatever Nvidia ships next.
Image: Sergei Starostin, via Pexels
