A Low-Power Actor-Critic Framework Based on Memristive Spiking Neural Network

2018 4th International Conference on Environmental Science and Material Application

生态环境科学;材料科学

Zhang, Yaozhong^1 ; Hu, Xiaofang^1^2 ; Zhou, Yue^2^3 ; Song, Wenbo^1

College of Computer and Information Science, Southwest University, Chongqing

400715, China^1

Brain-inspired Computing and Intelligent Control of Chongqing Key Lab, Chongqing

400715, China^2

College of Electronic and Information Engineering, Southwest University, Chongqing

400715, China^3

关键词: Actor-Critic methods;    Continuous control;    Encoding and decoding;    Energy efficient;    Learning rules;    Process information;    Spike timing dependent plasticities;    Spiking neural networks;   

Others  :  https://iopscience.iop.org/article/10.1088/1755-1315/252/3/032157/pdf
DOI  :  10.1088/1755-1315/252/3/032157

来源: IOP

【 摘 要 】

Traditional deep reinforcement learning (DRL) algorithms consume much energy. Energy-efficient spiking neural networks (SNNs) are promising technologies to bulid a low-power reinforcement learning architecture. In this paper, an actor-critic framework based on memrisitive SNN is proposed. To convey and process information in SNN, spike encoding and decoding systems are created. Then, an improved learning algorithm based on spike-timing-dependent plasticity (STDP) learning rule is designed to combine actor-critic method with SNN. Moreover, this learning algorithm is also hardware-friendly. Besides, memristive synapse is designed to accelerate this learning algorithm. Finally, a continuous control problem is applied to illustrate the effectiveness of the proposed framework. The results show the proposed framework is prior to traditional methods.