Study on Reconstruction and Feature Tracking of Silicone Heart 3D Surface

Sensors

Lirong Yin¹  Bo Yang²  Yan Liu²  Longhai Xiang²  Shan Liu²  Jiawei Tian²  Wenfeng Zheng²  Ziyan Zhang³ 

[1] Department of Geography and Anthropology, Louisiana State University, Baton Rouge, LA 70803, USA;School of Automation, University of Electronic Science and Technology of China, Chengdu 610054, China;School of Innovation and Entrepreneurship, Xi’an Fanyi University, Xi’an 710105, China;

关键词: Delaunay triangulation;    reconstruction of three-dimensional surface;    feature tracking;    convolutional neural network;   

DOI  :  10.3390/s21227570

来源: DOAJ

【 摘 要 】

At present, feature-based 3D reconstruction and tracking technology is widely applied in the medical field. In minimally invasive surgery, the surgeon can achieve three-dimensional reconstruction through the images obtained by the endoscope in the human body, restore the three-dimensional scene of the area to be operated on, and track the motion of the soft tissue surface. This enables doctors to have a clearer understanding of the location depth of the surgical area, greatly reducing the negative impact of 2D image defects and ensuring smooth operation. In this study, firstly, the 3D coordinates of each feature point are calculated by using the parameters of the parallel binocular endoscope and the spatial geometric constraints. At the same time, the discrete feature points are divided into multiple triangles using the Delaunay triangulation method. Then, the 3D coordinates of feature points and the division results of each triangle are combined to complete the 3D surface reconstruction. Combined with the feature matching method based on convolutional neural network, feature tracking is realized by calculating the three-dimensional coordinate changes of the same feature point in different frames. Finally, experiments are carried out on the endoscope image to complete the 3D surface reconstruction and feature tracking.

【 授权许可】

Unknown