OpenAI and Broadcom unveil Jalapeño: A custom AI chip built for LLM inference in record time

OpenAI and Broadcom have introduced Jalapeño, a custom-built inference processor tailored specifically for large language models (LLMs) and emerging agentic AI workloads. Unlike generic AI accelerators, this ASIC was designed from the ground up to address the unique demands of inference at scale, including data movement efficiency, compute-memory balance, and networking optimization.

The partnership marks a strategic shift for OpenAI, which views Jalapeño as the first in a series of purpose-built hardware solutions rather than a repurposed training accelerator or off-the-shelf GPU. By leveraging deep insights into LLM behavior, the team optimized the chip’s architecture around critical kernels, memory access patterns, and serving strategies to maximize throughput while minimizing latency—a critical advantage for reasoning and agent-based applications.

Built for performance, not speculation

OpenAI and Broadcom emphasize that Jalapeño is engineered to deliver higher performance per watt than existing state-of-the-art hardware. While the companies have not released specific benchmarks or performance targets, they claim the processor achieves utilization rates close to theoretical maximums, reducing wasted compute cycles and energy consumption. Early internal testing suggests significant efficiency gains over current solutions like AMD’s Instinct MI350-series and Nvidia’s Blackwell-based accelerators, though exact numbers remain undisclosed.

Key design choices reflect this focus on efficiency:

• Six HBM memory modules integrated directly into the package to minimize data movement delays.
• A massive compute chiplet measuring approximately 840 mm²—nearly matching the reticle size of advanced EUV lithography systems—packed with high-density compute resources.
• A floorplan optimized for repetitive, tiled architectures consistent with systolic-array or matrix-engine designs, though specifics remain unconfirmed.

Richard Ho, OpenAI’s hardware program lead, stated: "Jalapeño was designed from the ground up for LLM inference using detailed insights from our close collaboration with OpenAI researchers. We optimized the architecture around the kernels, memory movement, networking, and serving patterns that matter most for frontier AI models. Based on early testing, Jalapeño will efficiently execute our most important workloads close to the hardware’s theoretical limits."

From concept to silicon in nine months

The development cycle for Jalapeño was exceptionally fast—just nine months from initial design to tape-out—a timeline typically reserved for simpler chips. While Broadcom and OpenAI have not confirmed whether AI played a direct role in the chip’s design, they acknowledged using OpenAI’s models to accelerate parts of the optimization process. Traditionally, custom ASICs require 1.5 to 2 years to develop, making Jalapeño’s rapid deployment a notable achievement.

The chip’s packaging reveals a multi-chiplet design, with a large compute die surrounded by six HBM modules and additional chiplets likely handling input/output functions. This structure mirrors trends seen in AI training processors, where companies like AMD and Nvidia increasingly adopt multi-chiplet approaches to scale performance. OpenAI and Broadcom’s choice of a single, large compute chiplet may prioritize low-latency data access and reduced communication overhead between components.

What’s next for Jalapeño?

OpenAI plans to integrate Jalapeño into its infrastructure starting in late 2026, with engineering samples already operating in lab environments. The company is currently running workloads such as GPT-5.3-Codex-Spark, though no performance metrics or comparative benchmarks have been shared. While the processor’s potential to outperform existing hardware is promising, the lack of hard data leaves room for skepticism until real-world testing is completed.

As AI models grow more complex and agentic workloads demand faster, more efficient inference, custom hardware like Jalapeño could redefine the boundaries of what’s possible. Whether it becomes a benchmark for future designs—or remains a proof-of-concept—will depend on how OpenAI scales its deployment and how competitors respond with their own innovations.