Transferability and interpretability of the sepsis prediction models in the intensive care unit

BMC Medical Informatics and Decision Making

Research

Qiyu Chen¹  Wenlian Lu¹  Yaoqing Tang²  Ranran Li²  Hongping Qu²  Dechang Chen²  Lei Li²  ChihChe Lin³  Yaling Huang³  Chiming Lai³ 

[1] Department of Applied Mathematics, School of Mathematical Sciences, Fudan University, 200433, Shanghai, China;Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, 200025, Shanghai, China;Shanghai Electric Group Co., Ltd., Central Academe, Shanghai, China;

关键词: Sepsis;    Intensive care unit;    Machine learning;    Transfer learning;    Prognostication;    Model interpretability;   

DOI  :  10.1186/s12911-022-02090-3

 received in 2022-07-27, accepted in 2022-12-16,  发布年份 2022

来源: Springer

【 摘 要 】

BackgroundWe aimed to develop an early warning system for real-time sepsis prediction in the ICU by machine learning methods, with tools for interpretative analysis of the predictions. In particular, we focus on the deployment of the system in a target medical center with small historical samples.MethodsLight Gradient Boosting Machine (LightGBM) and multilayer perceptron (MLP) were trained on Medical Information Mart for Intensive Care (MIMIC-III) dataset and then finetuned on the private Historical Database of local Ruijin Hospital (HDRJH) using transfer learning technique. The Shapley Additive Explanations (SHAP) analysis was employed to characterize the feature importance in the prediction inference. Ultimately, the performance of the sepsis prediction system was further evaluated in the real-world study in the ICU of the target Ruijin Hospital.ResultsThe datasets comprised 6891 patients from MIMIC-III, 453 from HDRJH, and 67 from Ruijin real-world data. The area under the receiver operating characteristic curves (AUCs) for LightGBM and MLP models derived from MIMIC-III were 0.98 − 0.98 and 0.95 − 0.96 respectively on MIMIC-III dataset, and, in comparison, 0.82 − 0.86 and 0.84 − 0.87 respectively on HDRJH, from 1 to  5 h preceding. After transfer learning and ensemble learning, the AUCs of the final ensemble model were enhanced to 0.94 − 0.94 on HDRJH and to 0.86 − 0.9 in the real-world study in the ICU of the target Ruijin Hospital. In addition, the SHAP analysis illustrated the importance of age, antibiotics, net balance, and ventilation for sepsis prediction, making the model interpretable.ConclusionsOur machine learning model allows accurate real-time prediction of sepsis within 5-h preceding. Transfer learning can effectively improve the feasibility to deploy the prediction model in the target cohort, and ameliorate the model performance for external validation. SHAP analysis indicates that the role of antibiotic usage and fluid management needs further investigation. We argue that our system and methodology have the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention.Trial registration: NCT05088850 (retrospectively registered).

【 授权许可】

CC BY   
© The Author(s) 2022

【 图 表 】

12982_2022_119_Article_IEq154.gif

12982_2022_119_Article_IEq153.gif

12982_2022_119_Article_IEq139.gif

12888_2022_4365_Article_IEq28.gif

Fig. 1