Object segmentation in cluttered environment based on gaze tracing and gaze blinking

ROBOMECH Journal

Mitsuhiro Horade¹  Masaru Kojima²  Yasushi Mae³  Photchara Ratsamee⁴  Kazuto Kamiyama⁵  Tatsuo Arai⁶ 

[1] Department of Mechanical Systems Engineering, School of Systems Engineering, National Defense Academy, Tokyo, Japan;Graduate School of Engineering Science, Osaka University, Osaka, Japan;Graduate School of Engineering, Kansai University, Osaka, Japan;Graduate School of Information Science and Technology, Osaka University, Osaka, Japan;Takenaka Research & Development Institute, Takenaka Corporation, Tokyo, Japan;The University of Electro-Communications, Tokyo, Japan;

关键词: Gaze interface;    Human–robot interaction;    Object segmentation;   

DOI  :  10.1186/s40648-021-00214-4

来源: Springer

【 摘 要 】

People with disabilities, such as patients with motor paralysis conditions, lack independence and cannot move most parts of their bodies except for their eyes. Supportive robot technology is highly beneficial in supporting these types of patients. We propose a gaze-informed location-based (or gaze-based) object segmentation, which is a core module of successful patient-robot interaction in an object-search task (i.e., a situation when a robot has to search for and deliver a target object to the patient). We have introduced the concepts of gaze tracing (GT) and gaze blinking (GB), which are integrated into our proposed object segmentation technique, to yield the benefit of an accurate visual segmentation of unknown objects in a complex scene. Gaze tracing information can be used as a clue as to where the target object is located in a scene. Then, gaze blinking can be used to confirm the position of the target object. The effectiveness of our proposed method has been demonstrated using a humanoid robot in experiments with different types of highly cluttered scenes. Based on the limited gaze guidance from the user, we achieved an 85% F-score of unknown object segmentation in an unknown environment.

【 授权许可】

CC BY