macJNet: weakly-supervised multimodal image deformable registration using joint learning framework and multi-sampling cascaded MIND

BioMedical Engineering OnLine

Research

Jiansong Ji¹  Ben Hong²  Minglei Shen²  Xusheng Qian³  Zhiyong Zhou³  Jisu Hu³  Yakang Dai³ 

[1] Key Laboratory of Imaging Diagnosis and Minimally Invasive Intervention Research, The Fifth Affiliated Hospital of Wenzhou Medical University, Lishui, Zhejiang, China;School of Electronic and Optical Engineering, NanJing University of Science and Technology, Nanjing, Jiangsu, China;Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu, China;School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Suzhou, Jiangsu, China;

关键词: Deformable registration;    Multimodal;    Image descriptor;    Joint learning;    Semi-supervised segmentation;   

DOI  :  10.1186/s12938-023-01143-6

 received in 2023-01-10, accepted in 2023-07-27,  发布年份 2023

来源: Springer

【 摘 要 】

Deformable multimodal image registration plays a key role in medical image analysis. It remains a challenge to find accurate dense correspondences between multimodal images due to the significant intensity distortion and the large deformation. macJNet is proposed to align the multimodal medical images, which is a weakly-supervised multimodal image deformable registration method using a joint learning framework and multi-sampling cascaded modality independent neighborhood descriptor (macMIND). The joint learning framework consists of a multimodal image registration network and two segmentation networks. The proposed macMIND is a modality-independent image structure descriptor to provide dense correspondence for registration, which incorporates multi-orientation and multi-scale sampling patterns to build self-similarity context. It greatly enhances the representation ability of cross-modal features in the registration network. The semi-supervised segmentation networks generate anatomical labels to provide semantics correspondence for registration, and the registration network helps to improve the performance of multimodal image segmentation by providing the consistency of anatomical labels. 3D CT-MR liver image dataset with 118 samples is built for evaluation, and comprehensive experiments have been conducted to demonstrate that macJNet achieves superior performance over state-of-the-art multi-modality medical image registration methods.

【 授权许可】

CC BY   
© BioMed Central Ltd., part of Springer Nature 2023

【 图 表 】

13690_2023_1170_Article_IEq108.gif

12888_2023_5172_Article_IEq18.gif

Fig. 3

12888_2023_5172_Article_IEq15.gif

Fig. 3

12888_2023_5172_Article_IEq12.gif

12888_2023_5172_Article_IEq11.gif

Fig. 3

Fig. 2

Fig. 4

Fig. 2

Fig. 3

Fig. 2

Fig. 1