Weakly supervised segmentation for real-time surgical tool tracking

Healthcare Technology Letters

article

Eung-Joo Lee¹  William Plishker²  Xinyang Liu³  Shuvra S. Bhattacharyya¹  Raj Shekhar² 

[1] Department of Electrical and Computer Engineering and the Institute for Advanced Computer Studies, University of Maryland, College Park;Inc., College Park;Sheikh Zayed Institute for Pediatric Surgical Innovation, the Children's National Medical Center

关键词: endoscopes;    computer vision;    learning (artificial intelligence);    surgery;    image segmentation;    medical image processing;    medical robotics;    image sequences;    tracking;    neural nets;    annotated training data;    surgical tool segmentation;    laparoscopic image processing;    light-weight deep segmentation network;    binary segmentation mask;    automatic tool segmentation;    robust tool tracking;    supervised segmentation;    real-time surgical tool tracking;    surgical scenarios;    electromagnetic tracking;    vision-based methods;    tracking robustness;    deep learning-based methods;    pixel-wise training data;   

DOI  :  10.1049/htl.2019.0083

学科分类：肠胃与肝脏病学

来源: Wiley

【 摘 要 】

Surgical tool tracking has a variety of applications in different surgical scenarios. Electromagnetic (EM) tracking can be utilised for tool tracking, but the accuracy is often limited by magnetic interference. Vision-based methods have also been suggested; however, tracking robustness is limited by specular reflection, occlusions, and blurriness observed in the endoscopic image. Recently, deep learning-based methods have shown competitive performance on segmentation and tracking of surgical tools. The main bottleneck of these methods lies in acquiring a sufficient amount of pixel-wise, annotated training data, which demands substantial labour costs. To tackle this issue, the authors propose a weakly supervised method for surgical tool segmentation and tracking based on hybrid sensor systems. They first generate semantic labellings using EM tracking and laparoscopic image processing concurrently. They then train a light-weight deep segmentation network to obtain a binary segmentation mask that enables tool tracking. To the authors’ knowledge, the proposed method is the first to integrate EM tracking and laparoscopic image processing for generation of training labels. They demonstrate that their framework achieves accurate, automatic tool segmentation (i.e. without any manual labelling of the surgical tool to be tracked) and robust tool tracking in laparoscopic image sequences.

【 授权许可】

CC BY|CC BY-ND|CC BY-NC|CC BY-NC-ND