Vision-based markerless registration using stereo vision and an augmented reality surgical navigation system: a pilot study

BMC Medical Imaging

Research Article

Hongen Liao¹  Takeyoshi Dohi²  Huy Hoang Tran³  Ken Masamune⁴  Kazuto Hoshi⁵  Tsuyoshi Takato⁵  Hideyuki Suenaga⁵ 

[1] Department of Bioengineering, Graduate School of Engineering, The University of Tokyo, Tokyo, Japan;Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China;Department of Mechanical Engineering, School of Engineering, Tokyo Denki University, Tokyo, Japan;Department of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan;Department of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan;Faculty of Advanced Technology and Surgery, Institute of Advanced Biomedical Engineering and Science, Tokyo Women’s Medical University, Tokyo, Japan;Department of Oral-Maxillofacial Surgery, Dentistry and Orthodontics, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo ku, 113 8656, Tokyo, Japan;

关键词: Augmented reality;    Integral videography;    Markerless registration;    Stereo vision;    Three-dimensional image;   

DOI  :  10.1186/s12880-015-0089-5

 received in 2015-04-23, accepted in 2015-10-09,  发布年份 2015

来源: Springer

【 摘 要 】

BackgroundThis study evaluated the use of an augmented reality navigation system that provides a markerless registration system using stereo vision in oral and maxillofacial surgery.MethodA feasibility study was performed on a subject, wherein a stereo camera was used for tracking and markerless registration. The computed tomography data obtained from the volunteer was used to create an integral videography image and a 3-dimensional rapid prototype model of the jaw. The overlay of the subject’s anatomic site and its 3D-IV image were displayed in real space using a 3D-AR display. Extraction of characteristic points and teeth matching were done using parallax images from two stereo cameras for patient-image registration.ResultsAccurate registration of the volunteer’s anatomy with IV stereoscopic images via image matching was done using the fully automated markerless system, which recognized the incisal edges of the teeth and captured information pertaining to their position with an average target registration error of < 1 mm. These 3D-CT images were then displayed in real space with high accuracy using AR. Even when the viewing position was changed, the 3D images could be observed as if they were floating in real space without using special glasses.ConclusionTeeth were successfully used for registration via 3D image (contour) matching. This system, without using references or fiducial markers, displayed 3D-CT images in real space with high accuracy. The system provided real-time markerless registration and 3D image matching via stereo vision, which, combined with AR, could have significant clinical applications.

【 授权许可】

CC BY   
© Suenaga et al. 2015