Development and external validation of a mixed-effects deep learning model to diagnose COVID-19 from CT imaging

Frontiers in Medicine

Medicine

Manhui Wang¹  Cliff Addison¹  Stu Franks²  Cristin Merritt²  Steve Messenger³  Maria Mackey³  Yitian Zhao⁴  Renrong Sun⁵  Caroline McCann⁶  Yalin Zheng⁷  Wenyue Zhu⁷  Joshua Bridge⁷  Yanda Meng⁷  Thomas Fitzmaurice⁸ 

[1]Advanced Research Computing, University of Liverpool, Liverpool, United Kingdom

[2]Alces Flight Limited, Bicester, United Kingdom

[3]Amazon Web Services, London, United Kingdom

[4]Cixi Institute of Biomedical Engineering, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, China

[5]Department of Radiology, Hubei Provincial Hospital of Integrated Chinese and Western Medicine, Hubei University of Chinese Medicine, Wuhan, China

[6]Department of Radiology, Liverpool Heart and Chest Hospital NHS Foundation Trust, Liverpool, United Kingdom

[7]Institute of Life Course and Medical Sciences, University of Liverpool, Liverpool, United Kingdom

[8]Institute of Life Course and Medical Sciences, University of Liverpool, Liverpool, United Kingdom

[9]Department of Respiratory Medicine, Liverpool Heart and Chest Hospital NHS Foundation Trust, Liverpool, United Kingdom

关键词: CT;    COVID-19;    deep learning;    diagnosis;    imaging;   

DOI  :  10.3389/fmed.2023.1113030

 received in 2022-11-30, accepted in 2023-08-08,  发布年份 2023

来源: Frontiers

【 摘 要 】

BackgroundThe automatic analysis of medical images has the potential improve diagnostic accuracy while reducing the strain on clinicians. Current methods analyzing 3D-like imaging data, such as computerized tomography imaging, often treat each image slice as individual slices. This may not be able to appropriately model the relationship between slices.MethodsOur proposed method utilizes a mixed-effects model within the deep learning framework to model the relationship between slices. We externally validated this method on a data set taken from a different country and compared our results against other proposed methods. We evaluated the discrimination, calibration, and clinical usefulness of our model using a range of measures. Finally, we carried out a sensitivity analysis to demonstrate our methods robustness to noise and missing data.ResultsIn the external geographic validation set our model showed excellent performance with an AUROC of 0.930 (95%CI: 0.914, 0.947), with a sensitivity and specificity, PPV, and NPV of 0.778 (0.720, 0.828), 0.882 (0.853, 0.908), 0.744 (0.686, 0.797), and 0.900 (0.872, 0.924) at the 0.5 probability cut-off point. Our model also maintained good calibration in the external validation dataset, while other methods showed poor calibration.ConclusionDeep learning can reduce stress on healthcare systems by automatically screening CT imaging for COVID-19. Our method showed improved generalizability in external validation compared to previous published methods. However, deep learning models must be robustly assessed using various performance measures and externally validated in each setting. In addition, best practice guidelines for developing and reporting predictive models are vital for the safe adoption of such models.

【 授权许可】

Unknown   
Copyright © 2023 Bridge, Meng, Zhu, Fitzmaurice, McCann, Addison, Wang, Merritt, Franks, Mackey, Messenger, Sun, Zhao and Zheng.