Contents
Yesterday OpenAI put real, independently checked numbers behind a rumor that had been circulating for over a year: it built its own inference chip, and the first-generation part is competitive with Nvidia’s best. At the Hot Chips conference, OpenAI shared benchmarks for “Jalapeño,” its custom ASIC developed with Broadcom, tested by SemiAnalysis using its public InferenceX suite. The headline claim is 1.5x to 1.9x more AI work per watt at peak throughput than the best commercially available systems, with 1.7x to 3.6x lower end-to-end latency. For a company whose entire cost structure is dominated by how many tokens it can serve per dollar of power, that is not a marketing footnote, it is the number that decides margin.
What actually got tested
SemiAnalysis ran three open models on Jalapeño hardware, GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, and verified a subset of the runs on-site rather than taking OpenAI’s numbers on faith. On GPT-OSS, Jalapeño hit roughly 1,400 tokens per second per user. On DeepSeek R1, it topped 700 tokens per second on a single concurrent request. For interactive workloads specifically, the reported gain climbs to 2.1x to 4.1x over the comparison systems, which matters more than the peak-throughput number if your product is a chat interface rather than a batch job.
Where the comparison gets honest
Two details in the SemiAnalysis writeup are worth more than the headline multiplier. First, Jalapeño posted these results without multi-token prediction or speculative decoding, techniques that some of the Nvidia comparison systems were already using. That means the gap was measured with OpenAI’s chip handicapped relative to its rivals, not the other way around. Second, SemiAnalysis points out that Blackwell is not actually the fair comparison, since Jalapeño and Nvidia’s newer Vera Rubin platform both use HBM4 memory. Even against Rubin, Jalapeño still delivers more output tokens per megawatt, though Rubin’s use of multi-token prediction closes most of the gap on total cost of ownership per token, where the two land roughly even.
The caveats matter too. Nvidia and AMD have already published numbers for larger models like DeepSeek V4 Pro and Kimi K3 that Jalapeño hasn’t been tested against yet, and while Rubin systems are shipping to customers now, Jalapeño reportedly hasn’t moved past engineering samples. OpenAI CFO Sarah Friar has framed the chip as complementary to, not a replacement for, its existing Nvidia, AMD, AWS, Cerebras, and CoreWeave deals. The more striking detail is the development timeline: design work started in mid-2024, and the chip went from initial team to finished blueprint at the fab in about nine months, an unusually fast cycle that SemiAnalysis credits partly to OpenAI using its own models to help with chip design and optimization work.
Play with the actual disclosed numbers below, then use the slider to see what OpenAI’s claimed per-watt range would mean for a real inference power bill.
Why I’m paying attention
Every team I’ve worked with on LLM infrastructure treats the inference bill as a fact of nature, something you optimize around with batching, quantization, and caching, but never something the hardware market might just hand you a step-function improvement on. What makes Jalapeño different from the usual chip announcement is who is disclosing it: OpenAI is simultaneously the buyer with the most inference volume in the industry and now a chip vendor, which means every claim it makes gets checked against the bill it would otherwise be paying Nvidia. SemiAnalysis’s most pointed observation isn’t about the chip at all, it’s that OpenAI brought up production models on entirely new silicon in nine months using its own models to help with the chip design, which is the kind of thing that makes Nvidia’s CUDA software moat look less permanent than the stock price assumes.
None of this means Blackwell or Rubin get less relevant tomorrow, Jalapeño is still engineering samples while Rubin ships to customers today, and OpenAI’s own CFO describes the chip as additive to its Nvidia and AMD deals, not a replacement. But if you are the one signing off on GPU capacity for a production inference workload, the practical takeaway is to stop assuming today’s dollar-per-token number is a floor. Watch for Jalapeño showing up on larger models like DeepSeek V4 Pro or Kimi K3, and watch whether OpenAI starts selling inference capacity built on its own silicon rather than keeping it entirely internal. That is the point where this stops being a chip-design story and starts being a pricing story for everyone else running inference at scale.