Adaptive Q-learning Algorithm for AUV Route Planning

2019 2nd International Conference on Advanced Materials, Intelligent Manufacturing and Automation

Zhu, Yingying^1 ; Dai, Changan^2 ; Xia, Jiahao^3 ; Wu, Haowei^4 ; An, Li^3

Basic Teaching and Research Department of Command and Training, Armed Police Special Police Academy, Beijing

102211, China^1

China National Aeronautical Radio Electronics Research Institute Department, Shanghai

200233, China^2

Science and Technology on Underwater Vehicles Laboratory, Harbin Engineering University, Harbin

150001, China^3

Beijing Institute of Astronautically Systems Engineering, Beijing

100076, China^4

关键词: Change trends;    Environmental complexity;    Exploration and exploitation;    Learning convergence;    Learning environments;    Local optimal solution;    Q-learning algorithms;    Route planning;   

Others  :  https://iopscience.iop.org/article/10.1088/1757-899X/569/5/052029/pdf
DOI  :  10.1088/1757-899X/569/5/052029

来源: IOP

【 摘 要 】

When applying Q-learning algorithm to AUV route planning, it is difficult to strike a balance between exploration and exploitation. Usually, the fixed greedy rate or the greedy rate with fixed change trend is adopted according to experience. However, the value of greedy rate in the above method cannot be matched with the learning environment, and often occurs a problem of falling into a local optimal solution or slow learning convergence. In order to solve the above problems, we proposed an adaptive Q-learning algorithm. It guides the greedy rate with environmental complexity, and the environmental complexity degree of AUV is evaluated by the environmental complexity mathematical model. By setting up the simulation experiment of two-dimensional environment, the fixed greedy rate and the greedy rate with fixed change trend is compared. The simulation results show that the adaptive Q-learning algorithm considering environmental complexity converges faster and is less prone to fall into local optimal solutions.