In Situ Stochastic Training of MTJ Crossbars With Machine Learning Algorithms

ACM JOURNAL ON EMERGING TECHNOLOGIES IN COMPUTING SYSTEMS

Article; Proceedings Paper

Mondal, Ankit¹  Srivastava, Ankur¹ 

[1] Univ Maryland, Dept Elect & Comp Engn, 8223 Paint Branch Dr,Room 1313,Williams Bldg, College Pk, MD 20742 USA.

关键词: Magnetic tunnel junctions;    neural networks;    restricted Boltzmannmachines;    crossbar architecture;    on-chip learning;    GRADIENT DESCENT;    NEURAL-NETWORKS;   

DOI  :  10.1145/3309880

来源: SCIE

【 摘 要 】

Owing to high device density, scalability, and non-volatility, magnetic tunnel junction (MTJ)-based crossbars have garnered significant interest for implementing the weights of neural networks (NNs). The existence of only two stable states in MTJs implies a high overhead of obtaining optimal binary weights in software. This article illustrates that the inherent parallelism in the crossbar structure makes it highly appropriate for in situ training, wherein the network is taught directly on the hardware. It leads to significantly smaller training overhead as the training time is independent of the size of the network, while also circumventing the effects of alternate current paths in the crossbar and accounting for manufacturing variations in the device. We show how the stochastic switching characteristics of MTJs can be leveraged to perform probabilistic weight updates using the gradient descent algorithm. We describe how the update operations can be performed on crossbars implementing NNs and restricted Boltzmann machines, and perform simulations on them to demonstrate the effectiveness of our techniques. The results reveal that stochastically trained MTJ-crossbar feed-forward and deep belief nets achieve a classification accuracy nearly the same as that of real-valued weight networks trained in software and exhibit immunity to device variations.

【 授权许可】

Free