Compound Heuristic Information Guided Policy Improvement for Robot Motor Skill Acquisition

Applied Sciences

Fan Luo¹  Jian Fu¹  Xiang Teng¹  Cong Li¹  Boqun Li² 

[1] School of Automation, Wuhan University of Technology, Wuhan 430070, China;School of Electronic and Information Engineering, University of Science and Technology Liaoning, Anshan 114051, China;

关键词: robot learning;    reinforcement learning;    heuristic information;    KCCA;    PI²-CMA;   

DOI  :  10.3390/app10155346

来源: DOAJ

【 摘 要 】

Discovering the implicit pattern and using it as heuristic information to guide the policy search is one of the core factors to speed up the procedure of robot motor skill acquisition. This paper proposes a compound heuristic information guided reinforcement learning algorithm PI${}^2$-CMA-KCCA for policy improvement. Its structure and workflow are similar to a double closed-loop control system. The outer loop realized by Kernel Canonical Correlation Analysis (KCCA) infers the implicit nonlinear heuristic information between the joints of the robot. In addition, the inner loop operated by Covariance Matrix Adaptation (CMA) discovers the hidden linear correlations between the basis functions within the joint of the robot. These patterns which are good for learning the new task can automatically determine the mean and variance of the exploring perturbation for Path Integral Policy Improvement (PI${}^2$). Compared with classical PI², PI²-CMA, and PI²-KCCA, PI²-CMA-KCCA can not only endow the robot with the ability to realize transfer learning from the demonstration trajectory to the new task trajectory, but also complete it more efficiently. The classical via-point experiments based on SCARA and Swayer robots have validated that the proposed method has fast learning convergence and can find a solution for the new task.

【 授权许可】

Unknown