Calibration-Free Monocular Vision-Based Robot Manipulations With Occlusion Awareness

IEEE Access

Zhiyong Sun¹  Kun Dong¹  Lili Zhao¹  Bo Song¹  Yongle Luo¹  Erkang Cheng¹  Honglin Kan¹  Chao Zhou² 

[1] Chinese Academy of Sciences, Institute of Intelligent Machines, Hefei, China;Chinese Academy of Sciences, Institute of Plasma Physics, Hefei, China;

关键词: Monocular vision;    reinforcement learning;    reward shaping;    robot manipulation;   

DOI  :  10.1109/ACCESS.2021.3082947

来源: DOAJ

【 摘 要 】

Vision-based manipulation has been largely used in various robot applications. Normally, in order to obtain the spatial information of the operated target, a carefully calibrated stereo vision system is required. However, it limits the application of robots in the unstructured environment which limits both the number and the pose of the camera. In this study, a calibration-free monocular vision-based robot manipulation approach is proposed based on domain randomization and deep reinforcement learning (DRL). Firstly, a learning strategy combined domain randomization is developed to estimate the spatial information of the target from a single monocular camera arbitrarily mounted in a large area of the manipulation environment. Secondly, to address the monocular occlusion problem which regularly happens during robot manipulations, an occlusion awareness DRL policy has been designed to control the robot to avoid occlusions actively in the manipulation tasks. The performance of our method has been evaluated on two common manipulation tasks, reaching and lifting of a target building block, which show the efficiency and effectiveness of our proposed approach.

【 授权许可】

Unknown