Trained Biased Number Representation for ReRAM-Based Neural Network Accelerators

ACM JOURNAL ON EMERGING TECHNOLOGIES IN COMPUTING SYSTEMS

Article; Proceedings Paper

Wang, Weijia¹  Lin, Bill¹ 

[1] Univ Calif San Diego, Dept Elect & Comp Engn, La Jolla, CA 92093 USA.

关键词: Resistive Memory;    convolutional neural networks;    quantization;    machinelearning;    processing-in-memory;   

DOI  :  10.1145/3304107

来源: SCIE

【 摘 要 】

Recent works have demonstrated the promise of using resistive random access memory (ReRAM) to perform neural network computations in memory. In particular, ReRAM-based crossbar structures can perform matrix-vector multiplication directly in the analog domain, but the resolutions of ReRAM cells and digital/analog converters limit the precisions of inputs and weights that can be directly supported. Although convolutional neural networks (CNNs) can be trained with low-precision weights and activations, previous quantization approaches are either not amenable to ReRAM-based crossbar implementations or have poor accuracies when applied to deep CNNs on complex datasets. In this article, we propose a new CNN training and implementation approach that implements weights using a trained biased number representation, which can achieve near full-precision model accuracy with as little as 2-bit weights and 2-bit activations on the CIFAR datasets. The proposed approach is compatible with a ReRAM-based crossbar implementation. We also propose an activation-side coalescing technique that combines the steps of batch normalization, nonlinear activation, and quantization into a single stage that simply performs a clipped-rounding operation. Experiments demonstrate that our approach outperforms previous low-precision number representations for VGG-11, VGG-13, and VGG-19 models on both the CIFAR-10 and CIFAR-100 datasets.

【 授权许可】

Free