Linear fine-tuning: a linear transformation based transfer strategy for deep MRI reconstruction

Frontiers in Neuroscience

Neuroscience

Dayong Gao¹  Mengdie Song²  Jianan Xv²  Wanqing Bi²  Fulang Qi²  Xiaohan Hao³ 

[1] Department of Mechanical Engineering, University of Washington, Seattle, WA, United States;The Centers for Biomedical Engineering, University of Science and Technology of China, Hefei, Anhui, China;The Centers for Biomedical Engineering, University of Science and Technology of China, Hefei, Anhui, China;Fuqing Medical Co., Ltd., Hefei, Anhui, China;

关键词: magnetic resonance imaging reconstruction;    deep learning;    transfer learning;    fine-tuning;    transfer strategy;   

DOI  :  10.3389/fnins.2023.1202143

 received in 2023-04-07, accepted in 2023-06-05,  发布年份 2023

来源: Frontiers

【 摘 要 】

IntroductionFine-tuning (FT) is a generally adopted transfer learning method for deep learning-based magnetic resonance imaging (MRI) reconstruction. In this approach, the reconstruction model is initialized with pre-trained weights derived from a source domain with ample data and subsequently updated with limited data from the target domain. However, the direct full-weight update strategy can pose the risk of "catastrophic forgetting" and overfitting, hindering its effectiveness. The goal of this study is to develop a zero-weight update transfer strategy to preserve pre-trained generic knowledge and reduce overfitting.MethodsBased on the commonality between the source and target domains, we assume a linear transformation relationship of the optimal model weights from the source domain to the target domain. Accordingly, we propose a novel transfer strategy, linear fine-tuning (LFT), which introduces scaling and shifting (SS) factors into the pre-trained model. In contrast to FT, LFT only updates SS factors in the transfer phase, while the pre-trained weights remain fixed.ResultsTo evaluate the proposed LFT, we designed three different transfer scenarios and conducted a comparative analysis of FT, LFT, and other methods at various sampling rates and data volumes. In the transfer scenario between different contrasts, LFT outperforms typical transfer strategies at various sampling rates and considerably reduces artifacts on reconstructed images. In transfer scenarios between different slice directions or anatomical structures, LFT surpasses the FT method, particularly when the target domain contains a decreasing number of training images, with a maximum improvement of up to 2.06 dB (5.89%) in peak signal-to-noise ratio.DiscussionThe LFT strategy shows great potential to address the issues of "catastrophic forgetting" and overfitting in transfer scenarios for MRI reconstruction, while reducing the reliance on the amount of data in the target domain. Linear fine-tuning is expected to shorten the development cycle of reconstruction models for adapting complicated clinical scenarios, thereby enhancing the clinical applicability of deep MRI reconstruction.

【 授权许可】

Unknown   
Copyright © 2023 Bi, Xv, Song, Hao, Gao and Qi.