Reinforcement Learning-Based Cascade Motion Policy Design for Robust 3D Bipedal Locomotion

IEEE Access

Guillermo A. Castillo¹  Bowen Weng¹  Ayonga Hereid²  Wei Zhang³ 

[1] Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA;Department of Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH, USA;Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, China;

关键词: Motion control;    legged locomotion;    machine learning;   

DOI  :  10.1109/ACCESS.2022.3151771

来源: DOAJ

【 摘 要 】

This paper presents a novel reinforcement learning (RL) framework to design cascade feedback control policies for 3D bipedal locomotion. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint or task space trajectories. Unlike these studies, we propose a policy structure that decouples the bipedal locomotion problem into two modules that incorporate the physical insights from the nature of the walking dynamics and the well-established Hybrid Zero Dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, sample efficiency, and less dependence on prior knowledge. The proposed solution learns stable and robust walking gaits from scratch and allows the controller to realize omnidirectional walking with accurate tracking of the desired velocity and heading angle. The learned policies also perform robustly against various adversarial forces applied to the torso and walking blindly on a series of challenging and unstructured terrains. These results demonstrate that the proposed cascade feedback control policy is suitable for navigation of 3D bipedal robots in indoor and outdoor environments.

【 授权许可】

Unknown