A touch-free human-robot collaborative surgical navigation robotic system based on hand gesture recognition

Frontiers in Neuroscience

Neuroscience

Jie Wang¹  Xinkang Zhang¹  Xinrong Chen¹  Zhijian Song² 

[1] Academy for Engineering and Technology, Fudan University, Shanghai, China;Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, China;Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, China;Digital Medical Research Center, School of Basic Medical Science, Fudan University, Shanghai, China;

关键词: robot-assisted minimally invasive surgery;    surgical robot;    human-robot interaction;    gesture recognition;    contactless surgery;   

DOI  :  10.3389/fnins.2023.1200576

 received in 2023-04-05, accepted in 2023-05-15,  发布年份 2023

来源: Frontiers

【 摘 要 】

Robot-assisted minimally invasive surgery (RAMIS) has gained significant traction in clinical practice in recent years. However, most surgical robots rely on touch-based human-robot interaction (HRI), which increases the risk of bacterial diffusion. This risk is particularly concerning when surgeons must operate various equipment with their bare hands, necessitating repeated sterilization. Thus, achieving touch-free and precise manipulation with a surgical robot is challenging. To address this challenge, we propose a novel HRI interface based on gesture recognition, leveraging hand-keypoint regression and hand-shape reconstruction methods. By encoding the 21 keypoints from the recognized hand gesture, the robot can successfully perform the corresponding action according to predefined rules, which enables the robot to perform fine-tuning of surgical instruments without the need for physical contact with the surgeon. We evaluated the surgical applicability of the proposed system through both phantom and cadaver studies. In the phantom experiment, the average needle tip location error was 0.51  mm, and the mean angle error was 0.34 degrees. In the simulated nasopharyngeal carcinoma biopsy experiment, the needle insertion error was 0.16  mm, and the angle error was 0.10 degrees. These results indicate that the proposed system achieves clinically acceptable accuracy and can assist surgeons in performing contactless surgery with hand gesture interaction.

【 授权许可】

Unknown   
Copyright © 2023 Wang, Zhang, Chen and Song.