Novel Multimodal, Multiscale Imaging System with Augmented Reality

Diagnostics

Francis Papay¹  Christopher Mela²  Yang Liu² 

[1] Dermatology & Plastic Surgery Institute, Cleveland Clinic, Cleveland, OH 44195, USA;University of Iowa Technology Institute, The University of Iowa, 103 South Capitol Street, Iowa City, IA 52242, USA;

关键词: multimodal imaging;    multiscale imaging;    computer vision;    augmented reality;    fluorescence imaging;   

DOI  :  10.3390/diagnostics11030441

来源: DOAJ

【 摘 要 】

A novel multimodal, multiscale imaging system with augmented reality capability were developed and characterized. The system offers 3D color reflectance imaging, 3D fluorescence imaging, and augmented reality in real time. Multiscale fluorescence imaging was enabled by developing and integrating an in vivo fiber-optic microscope. Real-time ultrasound-fluorescence multimodal imaging used optically tracked fiducial markers for registration. Tomographical data are also incorporated using optically tracked fiducial markers for registration. Furthermore, we characterized system performance and registration accuracy in a benchtop setting. The multiscale fluorescence imaging facilitated assessing the functional status of tissues, extending the minimal resolution of fluorescence imaging to ~17.5 µm. The system achieved a mean of Target Registration error of less than 2 mm for registering fluorescence images to ultrasound images and MRI-based 3D model, which is within clinically acceptable range. The low latency and high frame rate of the prototype system has shown the promise of applying the reported techniques in clinically relevant settings in the future.

【 授权许可】

Unknown