Model-Free Recurrent Reinforcement Learning for AUV Horizontal Control

3rd International Conference on Automation, Control and Robotics Engineering

工业技术;计算机科学;无线电电子学

Huo, Yujia^1,2 ; Li, Yiping^1 ; Feng, Xisheng^1

State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang

110016, China^1

University of Chinese, Academy of Sciences, Beijing

100049, China^2

关键词: Autonomous underwater vehicles (AUV);    Continuous control;    Control simulation;    Dynamic information;    Partially observable Markov decision process;    Recurrent reinforcement learning;    Trajectory tracking;    Trajectory tracking tasks;   

Others  :  https://iopscience.iop.org/article/10.1088/1757-899X/428/1/012063/pdf
DOI  :  10.1088/1757-899X/428/1/012063

来源: IOP

【 摘 要 】

In this paper, aiming at the problems of 2-DOF horizontal motion control with high precision for autonomous underwater vehicle(AUV) trajectory tracking tasks, deep reinforcement learning controllers are applied to these conditions. These control problems are considered as a POMDP (Partially Observable Markov Decision Process). Model-free reinforcement learning(RL) algorithms for continuous control mission based on Deterministic Policy Gradient(DPG) allows robots learn from received delayed rewards when interacting with environments. Recurrent neural networks LSTM (Long Short-Term Memory) are involved into the reinforcement learning algorithm. Through this deep reinforcement learning algorithm, AUVs learn from sequences of dynamic information. The horizontal trajectory tracking tasks are described by LOS method and the motion control are idealized as a SISO model. Tanh-estimators are presented as data normalization. Moreover, AUV horizontal trajectory tracking and motion control simulation results demonstrate this algorithm gets better accuracy compared with the PID method and other non-recurrent methods. Efforts show the efficiency and effectiveness of the improved deep reinforcement learning algorithm.