Marker-less real-time intra-operative camera and hand-eye calibration procedure for surgical augmented reality

Healthcare Technology Letters

article

Megha Kalia¹  Prateek Mathur¹  Nassir Navab²  Septimiu E. Salcudean¹ 

[1] Robotics and Control Lab, Electrical and Computer Engineering, University of British Columbia;Technical University of Munich

关键词: medical robotics;    medical image processing;    cameras;    endoscopes;    calibration;    robot vision;    surgery;    phantoms;    rendering (computer graphics);    augmented reality;    image registration;    biomedical optical imaging;    hand-eye calibration procedure;    surgical augmented reality;    pre-operative medical data;    camera intrinsic matrix estimation;    hand-eye transformation;    camera calibration;    endoscope;    high visual error;    virtual rendered tool tip;    marker-less real-time intra-operative camera;    augmented reality rendering;    subsequent gradient descent steps;    da Vinci robot;    average target registration error;    prostate phantom;   

DOI  :  10.1049/htl.2019.0094

学科分类：肠胃与肝脏病学

来源: Wiley

【 摘 要 】

Accurate medical Augmented Reality (AR) rendering requires two calibrations, a camera intrinsic matrix estimation and a hand-eye transformation. We present a unified, practical, marker-less, real-time system to estimate both these transformations during surgery. For camera calibration we perform calibrations at multiple distances from the endoscope, pre-operatively, to parametrize the camera intrinsic matrix as a function of distance from the endoscope. Then, we retrieve the camera parameters intra-operatively by estimating the distance of the surgical site from the endoscope in less than 1 s. Unlike in prior work, our method does not require the endoscope to be taken out of the patient; for the hand-eye calibration, as opposed to conventional methods that require the identification of a marker, we make use of a rendered tool-tip in 3D. As the surgeon moves the instrument and observes the offset between the actual and the rendered tool-tip, they can select points of high visual error and manually bring the instrument tip to match the virtual rendered tool tip. To evaluate the hand-eye calibration, 5 subjects carried out the hand-eye calibration procedure on a da Vinci robot. Average Target Registration Error of approximately 7mm was achieved with just three data points.

【 授权许可】

CC BY|CC BY-ND|CC BY-NC|CC BY-NC-ND