ACM JOURNAL ON EMERGING TECHNOLOGIES IN COMPUTING SYSTEMS | |
Trained Biased Number Representation for ReRAM-Based Neural Network Accelerators | |
Article; Proceedings Paper | |
Wang, Weijia1  Lin, Bill1  | |
[1] Univ Calif San Diego, Dept Elect & Comp Engn, La Jolla, CA 92093 USA. | |
关键词: Resistive Memory; convolutional neural networks; quantization; machinelearning; processing-in-memory; | |
DOI : 10.1145/3304107 | |
来源: SCIE | |
【 摘 要 】
Recent works have demonstrated the promise of using resistive random access memory (ReRAM) to perform neural network computations in memory. In particular, ReRAM-based crossbar structures can perform matrix-vector multiplication directly in the analog domain, but the resolutions of ReRAM cells and digital/analog converters limit the precisions of inputs and weights that can be directly supported. Although convolutional neural networks (CNNs) can be trained with low-precision weights and activations, previous quantization approaches are either not amenable to ReRAM-based crossbar implementations or have poor accuracies when applied to deep CNNs on complex datasets. In this article, we propose a new CNN training and implementation approach that implements weights using a trained biased number representation, which can achieve near full-precision model accuracy with as little as 2-bit weights and 2-bit activations on the CIFAR datasets. The proposed approach is compatible with a ReRAM-based crossbar implementation. We also propose an activation-side coalescing technique that combines the steps of batch normalization, nonlinear activation, and quantization into a single stage that simply performs a clipped-rounding operation. Experiments demonstrate that our approach outperforms previous low-precision number representations for VGG-11, VGG-13, and VGG-19 models on both the CIFAR-10 and CIFAR-100 datasets.
【 授权许可】
Free
【 预 览 】
Files | Size | Format | View |
---|---|---|---|
RO202303098598807ZK.pdf | 2108KB | download |
【 参考文献 】
- [1]
- [2]
- [3]
- [4]
- [5]
- [6]
- [7]
- [8]
- [9]
- [10]
- [11]
- [12]
- [13]
- [14]
- [15]
- [16]
- [17]
- [18]
- [19]
- [20]
- [21]
- [22]
- [23]
- [24]
- [25]
- [26]
- [27]
- [28]