Memristor-based edge AI chip cuts power by 90%—but performance lags

Researchers from SK hynix, TetraMem, and the University of Southern California have unveiled an experimental system-on-chip (SoC) that leverages memristor-based in-memory computing (IMC) to accelerate neural network inference for edge AI devices. The prototype, designed primarily for lightweight models like MobileNet, aims to slash power consumption by performing computations directly within memory arrays—eliminating the data transfer bottlenecks that plague traditional architectures.

A hybrid approach to depthwise convolution

The core innovation lies in a dedicated depthwise convolution (DWC) accelerator integrated into the SoC. Unlike standard IMC designs that struggle with DWC operations—common in models such as MobileNet—the new architecture replaces conventional crossbar arrays with a zig-zag topology. This redesign allows eight specialized 252 × 28 memristor blocks to execute 28 parallel 3×3 convolutions simultaneously, maximizing crossbar utilization. The remaining nine neural processing units (NPUs) handle pointwise and dense layers using traditional IMC methods.

The SoC’s hybrid design combines analog and digital components:

• Nine standard NPUs: Each features a 256×256 memristor crossbar, 256 8-bit DACs (digital-to-analog converters), and 256 8-bit ADCs (analog-to-digital converters) to manage vector-matrix multiplications.
• One DWC-optimized NPU: Replaces the crossbar with eight zig-zag crossbar blocks, while retaining DACs/ADCs for signal conversion.
• RISC-V processor: Acts as the central scheduler, distributing workloads across the 10 NPUs.

Memristor devices were developed and fabricated by SK hynix using a 65nm CMOS back-end process, integrating resistive switching cells atop the existing circuitry.

Energy efficiency vs. performance trade-offs

In a demonstration using a customized MobileNetV1Small model for the Visual Wake Words benchmark, the SoC achieved an 80.36% inference accuracy—matching a 4-bit software baseline. However, performance metrics reveal significant limitations:

• Peak throughput: 0.254 TOPS per NPU, with a theoretical system-wide maximum of 2.54 TOPS.
• Energy efficiency: 21.3 TOPS/W at 100 MHz and 11.9 TOPS/W at 400 MHz.
• Hardware utilization: The MobileNet test used only six of the 10 NPUs (one DWC and five standard), leaving four idle. The authors did not validate whether all NPUs can operate simultaneously or how sustained throughput would scale under real-world conditions.

While the SoC outperforms some SRAM-based IMC accelerators in energy efficiency, its peak performance is 16× lower than Microsoft’s Copilot+ NPU requirements. The paper also claims a 10× advantage over Nvidia’s A100 in INT8 energy efficiency, though these comparisons lack independent verification.

A stepping stone for future edge AI hardware?

Despite its modest throughput, the project demonstrates a viable path forward for memristor-based IMC in edge devices. By addressing a critical bottleneck in depthwise convolution, the researchers have shown that analog computing can deliver competitive accuracy with extreme power savings. Yet, the SoC’s reliance on an outdated 65nm process and unproven scalability highlights the gap between proof-of-concept and commercial viability.

The next frontier may involve refining the architecture to support larger models or integrating it with advanced fabrication nodes. For now, this work serves as a reminder that innovation in edge AI hardware demands not just efficiency gains, but also scalable performance—two goals that often remain in tension.